[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:11 dist-test-slave-c12x krb5kdc[31150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:11 dist-test-slave-c12x krb5kdc[31150](info): set up 2 sockets
Aug 09 00:42:11 dist-test-slave-c12x krb5kdc[31150](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 00:42:12.425094 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:12 dist-test-slave-c12x krb5kdc[31150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:12.557101 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37669
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:12.648380 31170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:12.648597 31170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:12.648622 31170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:12.649866 31170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:12.649910 31170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:12.649925 31170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:12.649938 31170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:12.649987 31170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:12.651726 31170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:12.651996 31170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:12.652240 31170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:42:12.658191 31170 server_base.cc:1034] running on GCE node
W20260809 00:42:12.658154 31179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:12.658157 31175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:12.658157 31176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:12.658620 31170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:12.658953 31170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:12.660121 31170 hybrid_clock.cc:648] HybridClock initialized: now 1786236132660108 us; error 45 us; skew 500 ppm
Aug 09 00:42:12 dist-test-slave-c12x krb5kdc[31150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:12.663539 31170 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:12.664073 31170 webserver.cc:460] Webserver started at http://127.30.104.254:36565/ using document root <none> and password file <none>
I20260809 00:42:12.664345 31170 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:12.664417 31170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:12.664574 31170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:12.665958 31170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "f4b962dbc61b4511a265266f74f6f9fb"
format_stamp: "Formatted at 2026-08-09 00:42:12 on dist-test-slave-c12x"
server_key: "8b086c298c4632c05b6dc01f35e65d62"
server_key_iv: "ed29c14c4c81e0a3bf370e545da4ab46"
server_key_version: "encryptionkey@0"
I20260809 00:42:12.666390 31170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "f4b962dbc61b4511a265266f74f6f9fb"
format_stamp: "Formatted at 2026-08-09 00:42:12 on dist-test-slave-c12x"
server_key: "8b086c298c4632c05b6dc01f35e65d62"
server_key_iv: "ed29c14c4c81e0a3bf370e545da4ab46"
server_key_version: "encryptionkey@0"
I20260809 00:42:12.667985 31170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260809 00:42:12.668888 31187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:12.669126 31170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:12.669242 31170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "f4b962dbc61b4511a265266f74f6f9fb"
format_stamp: "Formatted at 2026-08-09 00:42:12 on dist-test-slave-c12x"
server_key: "8b086c298c4632c05b6dc01f35e65d62"
server_key_iv: "ed29c14c4c81e0a3bf370e545da4ab46"
server_key_version: "encryptionkey@0"
I20260809 00:42:12.669322 31170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:12.687661 31170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:12.690410 31170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:12.690645 31170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:12.696731 31170 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37669
I20260809 00:42:12.696772 31257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37669 every 8 connection(s)
I20260809 00:42:12.697309 31170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:12.698184 31258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:12.700773 31258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb: Bootstrap starting.
I20260809 00:42:12.701402 31258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:12.701704 31258 log.cc:824] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:12.702482 31258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb: No bootstrap required, opened a new log
I20260809 00:42:12.703758 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31170
I20260809 00:42:12.703876 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:12.704298 31258 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b962dbc61b4511a265266f74f6f9fb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37669 } }
I20260809 00:42:12.704447 31258 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:12.704471 31258 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4b962dbc61b4511a265266f74f6f9fb, State: Initialized, Role: FOLLOWER
I20260809 00:42:12.704620 31139 external_mini_cluster.cc:1428] Setting key a1224603a66c18ea7147ea351fcc7748
I20260809 00:42:12.704635 31258 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b962dbc61b4511a265266f74f6f9fb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37669 } }
I20260809 00:42:12.704695 31258 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:12.704749 31258 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:12.704800 31258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:12.706120 31258 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b962dbc61b4511a265266f74f6f9fb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37669 } }
I20260809 00:42:12.706246 31258 leader_election.cc:302] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4b962dbc61b4511a265266f74f6f9fb; no voters:
I20260809 00:42:12.706439 31258 leader_election.cc:288] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:12.706476 31266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:12.706704 31266 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [term 1 LEADER]: Becoming Leader. State: Replica: f4b962dbc61b4511a265266f74f6f9fb, State: Running, Role: LEADER
I20260809 00:42:12.706745 31258 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:12.706873 31266 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b962dbc61b4511a265266f74f6f9fb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37669 } }
I20260809 00:42:12.707401 31274 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4b962dbc61b4511a265266f74f6f9fb. Latest consensus state: current_term: 1 leader_uuid: "f4b962dbc61b4511a265266f74f6f9fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b962dbc61b4511a265266f74f6f9fb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37669 } } }
I20260809 00:42:12.707425 31272 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4b962dbc61b4511a265266f74f6f9fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4b962dbc61b4511a265266f74f6f9fb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37669 } } }
I20260809 00:42:12.707585 31274 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:12.707599 31272 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:12.709127 31293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:12.709849 31293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:12.711954 31293 catalog_manager.cc:1347] Generated new cluster ID: 3b532070ae0d4b178a9bc5af52a7c7a5
I20260809 00:42:12.712013 31293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:42:12 dist-test-slave-c12x krb5kdc[31150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236132, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:12.722613 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:12.705770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.30.104.254:37669)
0809 00:42:12.706179 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:12.706189 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:12.706230 (+ 41us) server_negotiation.cc:407] Connection header received
0809 00:42:12.709038 (+ 2808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:12.709046 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:12.709124 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:12.709279 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:12.711675 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:12.712808 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:12.714072 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:12.714522 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:12.714644 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:12.718818 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:12.718830 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:12.718830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:12.718843 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:12.720622 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:12.721092 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:12.721101 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:12.721101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:12.721204 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:12.721486 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:12.721487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:12.721487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:12.721739 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:12.721925 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:12.722208 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 00:42:12.722347 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":297,"threads_started":1}
I20260809 00:42:12.734668 31293 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:12.735409 31293 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:12.739149 31293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4b962dbc61b4511a265266f74f6f9fb: Generated new TSK 0
I20260809 00:42:12.739360 31293 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:12.770407 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37669
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:12.894181 31302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:12.894421 31302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:12.894454 31302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:12.896045 31302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:12.896106 31302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:12.896137 31302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:12.898239 31302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37669
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:12.898566 31302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:12.898854 31302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:12.905081 31314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:12.905278 31302 server_base.cc:1034] running on GCE node
W20260809 00:42:12.905157 31312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:12.905157 31311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:12.905673 31302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:12.905961 31302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:12.907160 31302 hybrid_clock.cc:648] HybridClock initialized: now 1786236132907145 us; error 52 us; skew 500 ppm
Aug 09 00:42:12 dist-test-slave-c12x krb5kdc[31150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:12.910797 31302 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:12.911306 31302 webserver.cc:460] Webserver started at http://127.30.104.193:46145/ using document root <none> and password file <none>
I20260809 00:42:12.911612 31302 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:12.911684 31302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:12.911849 31302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:12.913112 31302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "4ec082221bfd4ffe878645ecf4fcf94e"
format_stamp: "Formatted at 2026-08-09 00:42:12 on dist-test-slave-c12x"
server_key: "3b61f7783863278e5ea6414bbac196f8"
server_key_iv: "3ec6a5bb6385df2226d564c089522662"
server_key_version: "encryptionkey@0"
I20260809 00:42:12.913552 31302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "4ec082221bfd4ffe878645ecf4fcf94e"
format_stamp: "Formatted at 2026-08-09 00:42:12 on dist-test-slave-c12x"
server_key: "3b61f7783863278e5ea6414bbac196f8"
server_key_iv: "3ec6a5bb6385df2226d564c089522662"
server_key_version: "encryptionkey@0"
I20260809 00:42:12.915266 31302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:12.916267 31323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:12.916491 31302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:12.916710 31302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "4ec082221bfd4ffe878645ecf4fcf94e"
format_stamp: "Formatted at 2026-08-09 00:42:12 on dist-test-slave-c12x"
server_key: "3b61f7783863278e5ea6414bbac196f8"
server_key_iv: "3ec6a5bb6385df2226d564c089522662"
server_key_version: "encryptionkey@0"
I20260809 00:42:12.916795 31302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:12.925945 31302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:12.928598 31302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:12.928836 31302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:12.929178 31302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:12.929584 31302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:12.929646 31302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:12.929687 31302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:12.929715 31302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:12.944042 31302 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:33599
I20260809 00:42:12.944141 31479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:33599 every 8 connection(s)
I20260809 00:42:12.944545 31302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:12.948550 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31302
I20260809 00:42:12.948691 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:12.948865 31139 external_mini_cluster.cc:1428] Setting key 114bdd5212490da4748c6b6190ebbcd2
Aug 09 00:42:12 dist-test-slave-c12x krb5kdc[31150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:12.957304 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:12.945620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:51073 (local address 127.30.104.254:37669)
0809 00:42:12.945687 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:12.945689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:12.946008 (+ 319us) server_negotiation.cc:407] Connection header received
0809 00:42:12.946848 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:12.946851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:12.946864 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:12.946931 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:12.948142 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:12.948736 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:12.950080 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:12.950303 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:12.950402 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:12.953784 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:12.953786 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:12.953787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:12.953795 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:12.955694 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:12.956136 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:12.956139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:12.956139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:12.956214 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:12.956467 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:12.956468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:12.956468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:12.956648 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:12.956742 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:12.957059 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 00:42:12.957177 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:42:12.957450 31485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:12.945574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37669 (local address 127.30.104.193:51073)
0809 00:42:12.945791 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:12.945850 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:12.946720 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:12.947008 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:12.947012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:12.947766 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:12.948028 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:12.948034 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:12.948852 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:12.948852 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:12.949982 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:12.949985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:12.950408 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:12.950409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:12.950622 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:12.951289 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:12.951322 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:12.953640 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:12.955787 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:12.955795 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:12.955797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:12.956042 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:12.956288 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:12.956289 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:12.956289 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:12.956395 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:12.956761 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:12.956768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:12.956907 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 00:42:12.957011 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 00:42:12.957169 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":138,"threads_started":1}
I20260809 00:42:12.958325 31481 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37669
I20260809 00:42:12.958428 31481 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:12.958664 31481 heartbeater.cc:511] Master 127.30.104.254:37669 requested a full tablet report, sending...
I20260809 00:42:12.959389 31207 ts_manager.cc:194] Registered new tserver with Master: 4ec082221bfd4ffe878645ecf4fcf94e (127.30.104.193:33599)
I20260809 00:42:12.960469 31207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:51073
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:12.982735 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37669
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:13.100582 31489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:13.100816 31489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:13.100841 31489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:13.102119 31489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:13.102171 31489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:13.102200 31489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:13.103808 31489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37669
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:13.104136 31489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:13.104415 31489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:13.110015 31497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:13.109956 31498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:13.110653 31500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:13.111783 31489 server_base.cc:1034] running on GCE node
I20260809 00:42:13.112010 31489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:13.112277 31489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:13.113449 31489 hybrid_clock.cc:648] HybridClock initialized: now 1786236133113430 us; error 35 us; skew 500 ppm
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:13.116362 31489 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:13.116772 31489 webserver.cc:460] Webserver started at http://127.30.104.194:42305/ using document root <none> and password file <none>
I20260809 00:42:13.116995 31489 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:13.117046 31489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:13.117137 31489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:13.118201 31489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "e409e3f2f3e4442695760e2aec34a917"
format_stamp: "Formatted at 2026-08-09 00:42:13 on dist-test-slave-c12x"
server_key: "af2e56c4e3049a72fee17288f446cb79"
server_key_iv: "49574b306faee9f085092786e95cde25"
server_key_version: "encryptionkey@0"
I20260809 00:42:13.118530 31489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "e409e3f2f3e4442695760e2aec34a917"
format_stamp: "Formatted at 2026-08-09 00:42:13 on dist-test-slave-c12x"
server_key: "af2e56c4e3049a72fee17288f446cb79"
server_key_iv: "49574b306faee9f085092786e95cde25"
server_key_version: "encryptionkey@0"
I20260809 00:42:13.120188 31489 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:42:13.121030 31511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:13.121279 31489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:13.121395 31489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "e409e3f2f3e4442695760e2aec34a917"
format_stamp: "Formatted at 2026-08-09 00:42:13 on dist-test-slave-c12x"
server_key: "af2e56c4e3049a72fee17288f446cb79"
server_key_iv: "49574b306faee9f085092786e95cde25"
server_key_version: "encryptionkey@0"
I20260809 00:42:13.121485 31489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:13.133956 31489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:13.136372 31489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:13.136559 31489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:13.136833 31489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:13.137184 31489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:13.137250 31489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:13.137293 31489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:13.137321 31489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:13.145153 31489 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:46241
I20260809 00:42:13.145248 31671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:46241 every 8 connection(s)
I20260809 00:42:13.145593 31489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:13.149799 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31489
I20260809 00:42:13.149916 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:13.150086 31139 external_mini_cluster.cc:1428] Setting key 85047ceec92eb058d4cb58a2de6ce153
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:13.157383 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:13.146551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:55467 (local address 127.30.104.254:37669)
0809 00:42:13.146623 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:13.146625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:13.146954 (+ 329us) server_negotiation.cc:407] Connection header received
0809 00:42:13.147791 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:13.147792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:13.147804 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:13.147873 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:13.148943 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.149481 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.150894 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.151114 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.151173 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:13.153969 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:13.153971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:13.153972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:13.153981 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:13.155724 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:13.156295 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:13.156296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:13.156296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:13.156351 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:13.156661 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:13.156662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:13.156662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:13.156777 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:13.156834 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:13.157234 (+ 400us) server_negotiation.cc:299] Negotiation successful
0809 00:42:13.157314 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:42:13.157686 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:13.146548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37669 (local address 127.30.104.194:55467)
0809 00:42:13.146794 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:13.146833 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:13.147667 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:13.147961 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:13.147964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:13.148625 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:13.148831 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:13.148837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.149644 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.149645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:13.150795 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:13.150799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.151208 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.151209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:13.151365 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:13.151936 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:13.151947 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:13.153848 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:13.155817 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:13.155826 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:13.155829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:13.156210 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:13.156423 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:13.156424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:13.156424 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:13.156586 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:13.156854 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:13.156861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:13.157073 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 00:42:13.157213 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 00:42:13.157415 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":110,"threads_started":1}
I20260809 00:42:13.158202 31672 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37669
I20260809 00:42:13.158299 31672 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:13.158543 31672 heartbeater.cc:511] Master 127.30.104.254:37669 requested a full tablet report, sending...
I20260809 00:42:13.159003 31207 ts_manager.cc:194] Registered new tserver with Master: e409e3f2f3e4442695760e2aec34a917 (127.30.104.194:46241)
I20260809 00:42:13.159401 31207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:55467
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:13.178882 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37669
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:13.276145 31680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:13.276374 31680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:13.276402 31680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:13.277652 31680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:13.277700 31680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:13.277730 31680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:13.279413 31680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37669
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:13.279778 31680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:13.280035 31680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:13.285473 31688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:13.285521 31689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:13.285504 31692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:13.285715 31680 server_base.cc:1034] running on GCE node
I20260809 00:42:13.285956 31680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:13.286228 31680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:13.287503 31680 hybrid_clock.cc:648] HybridClock initialized: now 1786236133287426 us; error 45 us; skew 500 ppm
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:13.291260 31680 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:13.291867 31680 webserver.cc:460] Webserver started at http://127.30.104.195:40527/ using document root <none> and password file <none>
I20260809 00:42:13.292145 31680 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:13.292219 31680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:13.292392 31680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:13.293701 31680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "9ca69869f662415da07467ebb1a08611"
format_stamp: "Formatted at 2026-08-09 00:42:13 on dist-test-slave-c12x"
server_key: "e774984d787837497e7b9e0ff9d77aed"
server_key_iv: "95bc83d1d6ddb01cf5078c3ed776c077"
server_key_version: "encryptionkey@0"
I20260809 00:42:13.294164 31680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "9ca69869f662415da07467ebb1a08611"
format_stamp: "Formatted at 2026-08-09 00:42:13 on dist-test-slave-c12x"
server_key: "e774984d787837497e7b9e0ff9d77aed"
server_key_iv: "95bc83d1d6ddb01cf5078c3ed776c077"
server_key_version: "encryptionkey@0"
I20260809 00:42:13.296025 31680 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 00:42:13.297168 31701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:13.297495 31680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:13.297628 31680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "9ca69869f662415da07467ebb1a08611"
format_stamp: "Formatted at 2026-08-09 00:42:13 on dist-test-slave-c12x"
server_key: "e774984d787837497e7b9e0ff9d77aed"
server_key_iv: "95bc83d1d6ddb01cf5078c3ed776c077"
server_key_version: "encryptionkey@0"
I20260809 00:42:13.297735 31680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:13.317485 31680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:13.320317 31680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:13.320514 31680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:13.320794 31680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:13.321179 31680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:13.321214 31680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:13.321261 31680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:13.321290 31680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:13.334753 31680 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:37721
I20260809 00:42:13.334910 31862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:37721 every 8 connection(s)
I20260809 00:42:13.335250 31680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:13.335410 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31680
I20260809 00:42:13.335537 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:13.335731 31139 external_mini_cluster.cc:1428] Setting key cd5eb26752521d635451b425d3fd50c7
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:13.345714 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:13.336243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:57573 (local address 127.30.104.254:37669)
0809 00:42:13.336288 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:13.336291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:13.336755 (+ 464us) server_negotiation.cc:407] Connection header received
0809 00:42:13.337295 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:13.337296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:13.337308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:13.337348 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:13.338374 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.338883 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.340005 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.340215 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.340287 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:13.342608 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:13.342610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:13.342610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:13.342618 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:13.344203 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:13.344702 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:13.344703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:13.344704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:13.344775 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:13.345069 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:13.345070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:13.345070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:13.345196 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:13.345254 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:13.345570 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 00:42:13.345644 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:42:13.345904 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:13.336301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37669 (local address 127.30.104.195:57573)
0809 00:42:13.336603 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:13.336650 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:13.337185 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:13.337431 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:13.337434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:13.338066 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:13.338266 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:13.338271 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.338984 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.338985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:13.339899 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:13.339902 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.340308 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.340313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:13.340443 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:13.340939 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:13.340948 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:13.342492 (+ 1544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:13.344312 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:13.344318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:13.344321 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:13.344594 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:13.344869 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:13.344870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:13.344870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:13.344994 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:13.345286 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:13.345293 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:13.345437 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 00:42:13.345540 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 00:42:13.345676 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":170,"threads_started":1}
I20260809 00:42:13.346513 31863 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37669
I20260809 00:42:13.346745 31863 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:13.346972 31863 heartbeater.cc:511] Master 127.30.104.254:37669 requested a full tablet report, sending...
I20260809 00:42:13.347525 31207 ts_manager.cc:194] Registered new tserver with Master: 9ca69869f662415da07467ebb1a08611 (127.30.104.195:37721)
I20260809 00:42:13.347970 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:42:13.348001 31207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:57573
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236133, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:13.374974 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:13.365954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59200 (local address 127.30.104.193:33599)
0809 00:42:13.366264 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:13.366271 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:13.366285 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:42:13.366322 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:13.366324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:13.366348 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:13.366432 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:13.367303 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.368132 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.368921 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.369290 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.369380 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:13.371768 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:13.371777 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:13.371777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:13.371788 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:13.373634 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:13.374006 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:13.374009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:13.374010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:13.374074 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:13.374293 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:13.374294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:13.374294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:13.374447 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:13.374707 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:13.374757 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 00:42:13.374841 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":158,"threads_started":1}
W20260809 00:42:13.375391 31364 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59200
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:13.400684 31872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:13.393136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59204 (local address 127.30.104.193:33599)
0809 00:42:13.393206 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:13.393208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:13.393262 (+ 54us) server_negotiation.cc:407] Connection header received
0809 00:42:13.393314 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:13.393315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:13.393326 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:13.393378 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:13.393967 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.394483 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.395209 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:13.395424 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:13.395554 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:13.397992 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:13.397994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:13.397995 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:13.398006 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:13.399488 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:13.399856 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:13.399858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:13.399858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:13.399927 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:13.400132 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:13.400133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:13.400134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:13.400244 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:13.400364 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:13.400477 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:42:13.400566 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:42:13.401466 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31302
I20260809 00:42:13.413484 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31489
I20260809 00:42:13.423079 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31680
I20260809 00:42:13.434159 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31170
2026-08-09T00:42:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2256 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31881](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31881](info): set up 2 sockets
Aug 09 00:42:13 dist-test-slave-c12x krb5kdc[31881](info): commencing operation
krb5kdc: starting...
W20260809 00:42:14.674154 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:14 dist-test-slave-c12x krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:14.807410 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37111
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37111
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:14.891945 31901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:14.892202 31901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:14.892244 31901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:14.893406 31901 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:14.893482 31901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:14.893507 31901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:14.893528 31901 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:14.893560 31901 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:14.895115 31901 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:14.895407 31901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:14.895669 31901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:14.900915 31907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:14.900918 31911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:14.900918 31909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:14.901101 31901 server_base.cc:1034] running on GCE node
I20260809 00:42:14.901451 31901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:14.901798 31901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:14.902981 31901 hybrid_clock.cc:648] HybridClock initialized: now 1786236134902954 us; error 41 us; skew 500 ppm
Aug 09 00:42:14 dist-test-slave-c12x krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236134, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:14.905782 31901 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:14.906239 31901 webserver.cc:460] Webserver started at http://127.30.104.254:36695/ using document root <none> and password file <none>
I20260809 00:42:14.906503 31901 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:14.906574 31901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:14.906718 31901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:14.907975 31901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "fce76bdb9ef54fad9a6b490b2330ffbd"
format_stamp: "Formatted at 2026-08-09 00:42:14 on dist-test-slave-c12x"
server_key: "494781cffb920664a8023c32911a296d"
server_key_iv: "c526d711ecbc28450b735b6e6d0f6ee3"
server_key_version: "encryptionkey@0"
I20260809 00:42:14.908375 31901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "fce76bdb9ef54fad9a6b490b2330ffbd"
format_stamp: "Formatted at 2026-08-09 00:42:14 on dist-test-slave-c12x"
server_key: "494781cffb920664a8023c32911a296d"
server_key_iv: "c526d711ecbc28450b735b6e6d0f6ee3"
server_key_version: "encryptionkey@0"
I20260809 00:42:14.909904 31901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:14.910761 31922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:14.910957 31901 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:14.911068 31901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "fce76bdb9ef54fad9a6b490b2330ffbd"
format_stamp: "Formatted at 2026-08-09 00:42:14 on dist-test-slave-c12x"
server_key: "494781cffb920664a8023c32911a296d"
server_key_iv: "c526d711ecbc28450b735b6e6d0f6ee3"
server_key_version: "encryptionkey@0"
I20260809 00:42:14.911145 31901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:14.929471 31901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:14.932102 31901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:14.932283 31901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:14.936441 31901 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37111
I20260809 00:42:14.936563 32000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37111 every 8 connection(s)
I20260809 00:42:14.936892 31901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:14.937844 32001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:14.940377 32001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd: Bootstrap starting.
I20260809 00:42:14.941020 32001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:14.941336 32001 log.cc:824] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:14.942173 32001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd: No bootstrap required, opened a new log
I20260809 00:42:14.943848 32001 raft_consensus.cc:348] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fce76bdb9ef54fad9a6b490b2330ffbd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37111 } }
I20260809 00:42:14.944078 32001 raft_consensus.cc:374] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:14.944099 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31901
I20260809 00:42:14.944123 32001 raft_consensus.cc:729] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fce76bdb9ef54fad9a6b490b2330ffbd, State: Initialized, Role: FOLLOWER
I20260809 00:42:14.944183 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:14.944286 32001 consensus_queue.cc:260] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fce76bdb9ef54fad9a6b490b2330ffbd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37111 } }
I20260809 00:42:14.944355 31139 external_mini_cluster.cc:1428] Setting key 636dabe5d1b82c4e82281618bb300347
I20260809 00:42:14.944376 32001 raft_consensus.cc:388] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:14.944413 32001 raft_consensus.cc:482] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:14.944468 32001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:14.945637 32001 raft_consensus.cc:504] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fce76bdb9ef54fad9a6b490b2330ffbd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37111 } }
I20260809 00:42:14.945755 32001 leader_election.cc:302] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fce76bdb9ef54fad9a6b490b2330ffbd; no voters:
I20260809 00:42:14.945984 32001 leader_election.cc:288] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:14.946004 32007 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:14.946122 32007 raft_consensus.cc:686] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [term 1 LEADER]: Becoming Leader. State: Replica: fce76bdb9ef54fad9a6b490b2330ffbd, State: Running, Role: LEADER
I20260809 00:42:14.946266 32001 sys_catalog.cc:564] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:14.946273 32007 consensus_queue.cc:237] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fce76bdb9ef54fad9a6b490b2330ffbd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37111 } }
I20260809 00:42:14.947125 32009 sys_catalog.cc:455] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fce76bdb9ef54fad9a6b490b2330ffbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fce76bdb9ef54fad9a6b490b2330ffbd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37111 } } }
I20260809 00:42:14.947218 32009 sys_catalog.cc:458] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:14.947932 32008 sys_catalog.cc:455] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fce76bdb9ef54fad9a6b490b2330ffbd. Latest consensus state: current_term: 1 leader_uuid: "fce76bdb9ef54fad9a6b490b2330ffbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fce76bdb9ef54fad9a6b490b2330ffbd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37111 } } }
I20260809 00:42:14.948153 32008 sys_catalog.cc:458] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd [sys.catalog]: This master's current role is: LEADER
W20260809 00:42:14.948318 32027 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:42:14.948390 32027 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:42:14.948490 32023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:14.948662 32023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:14.950353 32023 catalog_manager.cc:1347] Generated new cluster ID: a64ea8adb146409eb77c45961bdf768f
I20260809 00:42:14.950415 32023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:42:14 dist-test-slave-c12x krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:14.956461 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:14.944893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46978 (local address 127.30.104.254:37111)
0809 00:42:14.945142 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:14.945154 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:14.945201 (+ 47us) server_negotiation.cc:407] Connection header received
0809 00:42:14.945356 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:14.945373 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:14.945432 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:14.945595 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:14.946369 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:14.947870 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:14.949034 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:14.949596 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:14.949735 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:14.952629 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:14.952633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:14.952634 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:14.952646 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:14.954664 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:14.955127 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:14.955130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:14.955131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:14.955252 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:14.955532 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:14.955532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:14.955533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:14.955838 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:14.955997 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:14.956122 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 00:42:14.956259 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":125,"threads_started":1}
I20260809 00:42:14.964519 32023 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:14.965109 32023 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:14.968931 32023 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fce76bdb9ef54fad9a6b490b2330ffbd: Generated new TSK 0
I20260809 00:42:14.969169 32023 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:15.002388 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37111
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:15.088487 32036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:15.088711 32036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:15.088735 32036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:15.089859 32036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:15.089898 32036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:15.089926 32036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:15.091379 32036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37111
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:15.091602 32036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:15.091790 32036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:15.096975 32043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:15.097061 32045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:15.096987 32049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:15.097570 32036 server_base.cc:1034] running on GCE node
I20260809 00:42:15.097810 32036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:15.098086 32036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:15.099246 32036 hybrid_clock.cc:648] HybridClock initialized: now 1786236135099233 us; error 31 us; skew 500 ppm
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:15.102089 32036 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:15.102500 32036 webserver.cc:460] Webserver started at http://127.30.104.193:35161/ using document root <none> and password file <none>
I20260809 00:42:15.102689 32036 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:15.102731 32036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:15.102823 32036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:15.103945 32036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "e445c0d7a5824cb3801a9165ea036347"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "f17ad19bc4ad940e7629700e87304893"
server_key_iv: "5191de7f09fb5d04d1745e52c05edd0b"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.104274 32036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "e445c0d7a5824cb3801a9165ea036347"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "f17ad19bc4ad940e7629700e87304893"
server_key_iv: "5191de7f09fb5d04d1745e52c05edd0b"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.105734 32036 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:15.106717 32057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.106957 32036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:15.107064 32036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "e445c0d7a5824cb3801a9165ea036347"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "f17ad19bc4ad940e7629700e87304893"
server_key_iv: "5191de7f09fb5d04d1745e52c05edd0b"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.107141 32036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:15.126744 32036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:15.129176 32036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:15.129343 32036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:15.129681 32036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:15.130033 32036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:15.130092 32036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.130131 32036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:15.130158 32036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.136153 32036 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:33225
I20260809 00:42:15.136206 32215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:33225 every 8 connection(s)
I20260809 00:42:15.136541 32036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:15.138235 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 32036
I20260809 00:42:15.138336 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:15.138511 31139 external_mini_cluster.cc:1428] Setting key db50fbb1ee87be245c035a24ad1a62b9
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
I20260809 00:42:15.150619 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.137253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:53449 (local address 127.30.104.254:37111)
0809 00:42:15.137315 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.137317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.137635 (+ 318us) server_negotiation.cc:407] Connection header received
0809 00:42:15.139004 (+ 1369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.139005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.139018 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.139101 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:15.141828 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.142350 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.144022 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.144247 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.144352 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.146952 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:15.146954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:15.146954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:15.146963 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:15.148979 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.149470 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.149472 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.149472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.149541 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.149852 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.149853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.149854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.149994 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:15.150083 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.150403 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.150513 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:42:15.150887 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.137224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37111 (local address 127.30.104.193:53449)
0809 00:42:15.137495 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:15.137546 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:15.138860 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:15.139220 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:15.139225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:15.141481 (+ 2256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:15.141698 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.141703 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.142470 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.142471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.143905 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.143908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.144366 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.144367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.144508 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.145091 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:15.145101 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:15.146825 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:15.149075 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:15.149083 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:15.149086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:15.149360 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:15.149642 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:15.149642 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:15.149643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:15.149771 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:15.150073 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:15.150080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:15.150244 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 00:42:15.150363 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 00:42:15.150560 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":97,"threads_started":1}
I20260809 00:42:15.151518 32216 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
I20260809 00:42:15.151628 32216 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:15.151889 32216 heartbeater.cc:511] Master 127.30.104.254:37111 requested a full tablet report, sending...
I20260809 00:42:15.152506 31945 ts_manager.cc:194] Registered new tserver with Master: e445c0d7a5824cb3801a9165ea036347 (127.30.104.193:33225)
I20260809 00:42:15.153211 31945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:53449
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:15.168737 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37111
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:15.253667 32225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:15.253894 32225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:15.253921 32225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:15.255036 32225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:15.255076 32225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:15.255106 32225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:15.256616 32225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37111
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:15.256811 32225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:15.256994 32225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:15.262457 32233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:15.262418 32235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:15.262403 32238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:15.262452 32225 server_base.cc:1034] running on GCE node
I20260809 00:42:15.262863 32225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:15.263154 32225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:15.264317 32225 hybrid_clock.cc:648] HybridClock initialized: now 1786236135264260 us; error 76 us; skew 500 ppm
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:15.267509 32225 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:15.268119 32225 webserver.cc:460] Webserver started at http://127.30.104.194:37419/ using document root <none> and password file <none>
I20260809 00:42:15.268397 32225 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:15.268473 32225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:15.268630 32225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:15.269901 32225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "dd79506eab214525bc239ab3b4bb2b8a"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "29662794a16e4fd6c97d3a3473e4a03a"
server_key_iv: "6a41a0e057d1ceeaf79203f10bead1d1"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.270294 32225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "dd79506eab214525bc239ab3b4bb2b8a"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "29662794a16e4fd6c97d3a3473e4a03a"
server_key_iv: "6a41a0e057d1ceeaf79203f10bead1d1"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.272076 32225 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 00:42:15.273010 32246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.273222 32225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:15.273366 32225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "dd79506eab214525bc239ab3b4bb2b8a"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "29662794a16e4fd6c97d3a3473e4a03a"
server_key_iv: "6a41a0e057d1ceeaf79203f10bead1d1"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.273475 32225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:15.298432 32225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:15.301316 32225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:15.301519 32225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:15.301820 32225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:15.302229 32225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:15.302294 32225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.302343 32225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:15.302371 32225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.309993 32225 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:46653
I20260809 00:42:15.310101 32421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:46653 every 8 connection(s)
I20260809 00:42:15.310480 32225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:15.315372 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 32225
I20260809 00:42:15.315562 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:15.315778 31139 external_mini_cluster.cc:1428] Setting key 034c0dbe8b4465fce357101e59ce8a10
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:15.321908 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.311472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:60079 (local address 127.30.104.254:37111)
0809 00:42:15.311528 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.311530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.311776 (+ 246us) server_negotiation.cc:407] Connection header received
0809 00:42:15.312762 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.312763 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.312775 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.312847 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:15.313638 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.314368 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.315759 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.316045 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.316138 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.318814 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:15.318817 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:15.318817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:15.318826 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:15.320386 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.320852 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.320853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.320854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.320921 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.321196 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.321197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.321197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.321330 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:15.321408 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.321708 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.321811 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:15.322122 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.311394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37111 (local address 127.30.104.194:60079)
0809 00:42:15.311610 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:15.311653 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:15.312630 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:15.312925 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:15.312929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:15.313223 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:15.313438 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.313442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.314572 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.314573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.315652 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.315655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.316132 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.316133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.316281 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.317180 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:15.317190 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:15.318695 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:15.320506 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:15.320514 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:15.320517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:15.320764 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:15.320994 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:15.320995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:15.320995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:15.321118 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:15.321399 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:15.321406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:15.321569 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 00:42:15.321688 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 00:42:15.321867 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":141,"threads_started":1}
I20260809 00:42:15.322691 32422 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37111
I20260809 00:42:15.322835 32422 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:15.323042 32422 heartbeater.cc:511] Master 127.30.104.254:37111 requested a full tablet report, sending...
I20260809 00:42:15.323545 31945 ts_manager.cc:194] Registered new tserver with Master: dd79506eab214525bc239ab3b4bb2b8a (127.30.104.194:46653)
I20260809 00:42:15.323959 31945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:60079
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:15.346731 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37111
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:15.433988 32428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:15.434216 32428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:15.434243 32428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:15.435384 32428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:15.435501 32428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:15.435576 32428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:15.437153 32428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37111
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:15.437372 32428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:15.437575 32428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:15.443172 32440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:15.443161 32437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:15.443195 32428 server_base.cc:1034] running on GCE node
W20260809 00:42:15.443161 32444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:15.443737 32428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:15.444041 32428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:15.445227 32428 hybrid_clock.cc:648] HybridClock initialized: now 1786236135445219 us; error 38 us; skew 500 ppm
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:15.448365 32428 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:15.448832 32428 webserver.cc:460] Webserver started at http://127.30.104.195:44593/ using document root <none> and password file <none>
I20260809 00:42:15.449115 32428 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:15.449200 32428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:15.449357 32428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:15.450457 32428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "ccde81d00a7144e68d82647b6725d676"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "c03ab83968c44df33977f26e348709d4"
server_key_iv: "32aa46589d95f4873ab7cee23e33f261"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.450837 32428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "ccde81d00a7144e68d82647b6725d676"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "c03ab83968c44df33977f26e348709d4"
server_key_iv: "32aa46589d95f4873ab7cee23e33f261"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.452323 32428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:15.453150 32453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.453346 32428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:15.453459 32428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "ccde81d00a7144e68d82647b6725d676"
format_stamp: "Formatted at 2026-08-09 00:42:15 on dist-test-slave-c12x"
server_key: "c03ab83968c44df33977f26e348709d4"
server_key_iv: "32aa46589d95f4873ab7cee23e33f261"
server_key_version: "encryptionkey@0"
I20260809 00:42:15.453547 32428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:15.474426 32428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:15.477182 32428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:15.477360 32428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:15.477619 32428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:15.478020 32428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:15.478084 32428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.478128 32428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:15.478156 32428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:15.484505 32428 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:42597
I20260809 00:42:15.484563 32613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:42597 every 8 connection(s)
I20260809 00:42:15.485136 32428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:15.493492 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 32428
I20260809 00:42:15.493604 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:15.493778 31139 external_mini_cluster.cc:1428] Setting key ea10921342ee67d9135dd8441ead23fe
I20260809 00:42:15.495579 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.485966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:46957 (local address 127.30.104.254:37111)
0809 00:42:15.486028 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.486030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.486251 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:42:15.487275 (+ 1024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.487277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.487291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.487343 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:15.488063 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.488593 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.489683 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.489902 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.489980 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.492497 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:15.492498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:15.492499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:15.492532 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:15.494075 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.494529 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.494530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.494530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.494595 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.494859 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.494860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.494860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.494991 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:15.495073 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.495354 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.495475 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:42:15.495776 32617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.485913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37111 (local address 127.30.104.195:46957)
0809 00:42:15.486127 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:15.486165 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:15.487145 (+ 980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:15.487422 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:15.487427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:15.487756 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:15.487957 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.487961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.488704 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.488705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.489573 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.489576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.490001 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.490002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.490143 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.490726 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:15.490735 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:15.492372 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:15.494168 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:15.494175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:15.494178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:15.494457 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:15.494672 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:15.494673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:15.494673 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:15.494791 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:15.495064 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:15.495070 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:15.495219 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 00:42:15.495331 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 00:42:15.495526 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":110,"threads_started":1}
I20260809 00:42:15.496263 32615 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37111
I20260809 00:42:15.496359 32615 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:15.496610 32615 heartbeater.cc:511] Master 127.30.104.254:37111 requested a full tablet report, sending...
I20260809 00:42:15.497099 31945 ts_manager.cc:194] Registered new tserver with Master: ccde81d00a7144e68d82647b6725d676 (127.30.104.195:42597)
I20260809 00:42:15.497465 31945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:46957
I20260809 00:42:15.506021 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:42:15.515270 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.507927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46986 (local address 127.30.104.254:37111)
0809 00:42:15.508027 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.508029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.508076 (+ 47us) server_negotiation.cc:407] Connection header received
0809 00:42:15.508131 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.508132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.508142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.508196 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:15.508789 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.509408 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.510309 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.510530 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.510601 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.512348 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:15.512350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:15.512350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:15.512358 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:15.513785 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.514213 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.514214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.514214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.514276 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.514534 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.514535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.514535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.514741 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:15.514838 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.515042 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.515161 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 00:42:15.518391 31945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:15.520200 31945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:15.527858 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.523165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.30.104.195:42597)
0809 00:42:15.523721 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.523727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.523739 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:42:15.523786 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.523789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.523806 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.524118 (+ 312us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:15.524210 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.524920 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.526525 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.527391 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.527521 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.527605 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.527717 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.527771 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":141,"threads_started":1}
I20260809 00:42:15.527851 32628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.523052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:42597 (local address 127.0.0.1:49710)
0809 00:42:15.523187 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:15.523232 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:15.523301 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:15.523925 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:15.523928 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:15.523931 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:15.524024 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.524026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.525029 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.525030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.526405 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.526409 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.527514 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.527515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.527607 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.527622 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:42:15.527699 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:42:15.527754 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":98,"thread_start_us":79,"threads_started":1}
I20260809 00:42:15.528000 32631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.523288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:46653 (local address 127.0.0.1:42238)
0809 00:42:15.523421 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:15.523455 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:15.523558 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:15.524130 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:15.524130 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:15.524132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:15.524170 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.524171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.525095 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.525096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.526498 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.526501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.527628 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.527629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.527713 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.527717 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:15.527771 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:42:15.527812 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":22,"thread_start_us":83,"threads_started":1}
I20260809 00:42:15.528249 32632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.523460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.30.104.194:46653)
0809 00:42:15.523764 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.523769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.523858 (+ 89us) server_negotiation.cc:407] Connection header received
0809 00:42:15.523933 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.523940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.523954 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.524058 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:15.524245 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.524956 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.526583 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.527393 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.527533 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.527597 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.527898 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.528155 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":105,"threads_started":1}
I20260809 00:42:15.528818 32521 tablet_service.cc:1467] Processing CreateTablet for tablet a8c98d23244241689febc6e118b8c5a8 (DEFAULT_TABLE table=test-table [id=6c5098c5bdf741c886d4c0a0725c8ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:15.529034 32324 tablet_service.cc:1467] Processing CreateTablet for tablet a8c98d23244241689febc6e118b8c5a8 (DEFAULT_TABLE table=test-table [id=6c5098c5bdf741c886d4c0a0725c8ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:15.529275 32521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c98d23244241689febc6e118b8c5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:15.529372 32324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c98d23244241689febc6e118b8c5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:15.532400 32634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.523715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50116 (local address 127.30.104.193:33225)
0809 00:42:15.525111 (+ 1396us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.525116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.525125 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:15.525168 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.525170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.525185 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.525266 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:15.525477 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.526520 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.527399 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.531853 (+ 4454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.531947 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.532013 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.532270 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.532317 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1354,"thread_start_us":86,"threads_started":1}
I20260809 00:42:15.532660 32633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.523578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33225 (local address 127.0.0.1:50116)
0809 00:42:15.523719 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:15.523728 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:15.523782 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:15.525288 (+ 1506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:15.525289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:15.525291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:15.525390 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.525392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.526626 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.526626 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.527299 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.527300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.532073 (+ 4773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.532074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.532135 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.532139 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:15.532247 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:42:15.532303 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":46,"thread_start_us":85,"threads_started":1}
I20260809 00:42:15.533407 32128 tablet_service.cc:1467] Processing CreateTablet for tablet a8c98d23244241689febc6e118b8c5a8 (DEFAULT_TABLE table=test-table [id=6c5098c5bdf741c886d4c0a0725c8ff5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:15.533867 32128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8c98d23244241689febc6e118b8c5a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:15.535586 32635 tablet_bootstrap.cc:492] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676: Bootstrap starting.
I20260809 00:42:15.536288 32635 tablet_bootstrap.cc:654] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:15.536854 32637 tablet_bootstrap.cc:492] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347: Bootstrap starting.
I20260809 00:42:15.536844 32635 log.cc:824] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:15.537235 32636 tablet_bootstrap.cc:492] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a: Bootstrap starting.
I20260809 00:42:15.537549 32637 tablet_bootstrap.cc:654] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:15.537844 32636 tablet_bootstrap.cc:654] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:15.538079 32637 log.cc:824] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:15.538357 32636 log.cc:824] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:15.538834 32637 tablet_bootstrap.cc:492] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347: No bootstrap required, opened a new log
I20260809 00:42:15.538892 32637 ts_tablet_manager.cc:1397] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:15.538966 32635 tablet_bootstrap.cc:492] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676: No bootstrap required, opened a new log
I20260809 00:42:15.539047 32635 ts_tablet_manager.cc:1397] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 00:42:15.539225 32636 tablet_bootstrap.cc:492] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a: No bootstrap required, opened a new log
I20260809 00:42:15.539294 32636 ts_tablet_manager.cc:1397] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:15.540711 32637 raft_consensus.cc:348] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.540886 32637 raft_consensus.cc:374] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:15.540940 32637 raft_consensus.cc:729] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e445c0d7a5824cb3801a9165ea036347, State: Initialized, Role: FOLLOWER
I20260809 00:42:15.541078 32637 consensus_queue.cc:260] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.541141 32635 raft_consensus.cc:348] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.541326 32635 raft_consensus.cc:374] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:15.541385 32637 ts_tablet_manager.cc:1428] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:42:15.541378 32635 raft_consensus.cc:729] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccde81d00a7144e68d82647b6725d676, State: Initialized, Role: FOLLOWER
I20260809 00:42:15.541369 32636 raft_consensus.cc:348] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.541529 32216 heartbeater.cc:503] Master 127.30.104.254:37111 was elected leader, sending a full tablet report...
I20260809 00:42:15.541532 32636 raft_consensus.cc:374] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:15.541572 32636 raft_consensus.cc:729] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd79506eab214525bc239ab3b4bb2b8a, State: Initialized, Role: FOLLOWER
I20260809 00:42:15.541536 32635 consensus_queue.cc:260] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.541680 32636 consensus_queue.cc:260] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.541821 32635 ts_tablet_manager.cc:1428] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 00:42:15.541955 32636 ts_tablet_manager.cc:1428] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:42:15.541952 32615 heartbeater.cc:503] Master 127.30.104.254:37111 was elected leader, sending a full tablet report...
I20260809 00:42:15.542064 32422 heartbeater.cc:503] Master 127.30.104.254:37111 was elected leader, sending a full tablet report...
I20260809 00:42:15.545027 32648 raft_consensus.cc:482] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:15.545137 32648 raft_consensus.cc:504] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.545509 32648 leader_election.cc:288] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccde81d00a7144e68d82647b6725d676 (127.30.104.195:42597), e445c0d7a5824cb3801a9165ea036347 (127.30.104.193:33225)
I20260809 00:42:15.548693 32425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.545674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:42597 (local address 127.30.104.194:55503)
0809 00:42:15.545735 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:15.545748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:15.545800 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:15.545918 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:15.545919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:15.545920 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:15.545994 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.545996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.546849 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.546849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.547803 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.547807 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.548397 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.548398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.548498 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.548501 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:15.548572 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:42:15.548613 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:42:15.548686 32630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.545757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:55503 (local address 127.30.104.195:42597)
0809 00:42:15.545821 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.545823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.545831 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:15.545857 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.545858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.545867 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.545917 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:15.546080 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.546672 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.547906 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.548309 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.548399 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.548409 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.548578 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.548619 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:15.548791 32634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.546005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:40243 (local address 127.30.104.193:33225)
0809 00:42:15.546044 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.546046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.546054 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:15.546075 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.546077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.546084 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.546133 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:15.546294 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.546934 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.548059 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.548430 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.548515 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.548523 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.548688 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.548726 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:42:15.548862 32649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.545775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33225 (local address 127.30.104.194:40243)
0809 00:42:15.545938 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:15.545950 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:15.546006 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:15.546135 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:15.546135 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:15.546136 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:15.546214 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.546215 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.547048 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.547049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.547971 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:15.547974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.548515 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.548516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:15.548596 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.548600 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:15.548678 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:42:15.548717 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":137,"threads_started":1}
I20260809 00:42:15.549355 32158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c98d23244241689febc6e118b8c5a8" candidate_uuid: "dd79506eab214525bc239ab3b4bb2b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e445c0d7a5824cb3801a9165ea036347" is_pre_election: true
I20260809 00:42:15.549504 32158 raft_consensus.cc:2454] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd79506eab214525bc239ab3b4bb2b8a in term 0.
I20260809 00:42:15.549758 32545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c98d23244241689febc6e118b8c5a8" candidate_uuid: "dd79506eab214525bc239ab3b4bb2b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccde81d00a7144e68d82647b6725d676" is_pre_election: true
I20260809 00:42:15.549805 32251 leader_election.cc:302] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd79506eab214525bc239ab3b4bb2b8a, e445c0d7a5824cb3801a9165ea036347; no voters:
I20260809 00:42:15.549914 32545 raft_consensus.cc:2454] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd79506eab214525bc239ab3b4bb2b8a in term 0.
I20260809 00:42:15.549989 32648 raft_consensus.cc:2781] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:15.550081 32648 raft_consensus.cc:482] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:15.550108 32648 raft_consensus.cc:3037] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:15.550809 32648 raft_consensus.cc:504] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.550974 32648 leader_election.cc:288] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [CANDIDATE]: Term 1 election: Requested vote from peers ccde81d00a7144e68d82647b6725d676 (127.30.104.195:42597), e445c0d7a5824cb3801a9165ea036347 (127.30.104.193:33225)
I20260809 00:42:15.551155 32545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c98d23244241689febc6e118b8c5a8" candidate_uuid: "dd79506eab214525bc239ab3b4bb2b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccde81d00a7144e68d82647b6725d676"
I20260809 00:42:15.551179 32158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8c98d23244241689febc6e118b8c5a8" candidate_uuid: "dd79506eab214525bc239ab3b4bb2b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e445c0d7a5824cb3801a9165ea036347"
I20260809 00:42:15.551256 32545 raft_consensus.cc:3037] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:15.551285 32158 raft_consensus.cc:3037] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:15.552163 32545 raft_consensus.cc:2454] T a8c98d23244241689febc6e118b8c5a8 P ccde81d00a7144e68d82647b6725d676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd79506eab214525bc239ab3b4bb2b8a in term 1.
I20260809 00:42:15.552167 32158 raft_consensus.cc:2454] T a8c98d23244241689febc6e118b8c5a8 P e445c0d7a5824cb3801a9165ea036347 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd79506eab214525bc239ab3b4bb2b8a in term 1.
I20260809 00:42:15.552331 32251 leader_election.cc:302] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccde81d00a7144e68d82647b6725d676, dd79506eab214525bc239ab3b4bb2b8a; no voters:
I20260809 00:42:15.552443 32648 raft_consensus.cc:2781] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:15.552635 32648 raft_consensus.cc:686] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [term 1 LEADER]: Becoming Leader. State: Replica: dd79506eab214525bc239ab3b4bb2b8a, State: Running, Role: LEADER
I20260809 00:42:15.552769 32648 consensus_queue.cc:237] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } }
I20260809 00:42:15.553694 31942 catalog_manager.cc:5495] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to dd79506eab214525bc239ab3b4bb2b8a (127.30.104.194). New cstate: current_term: 1 leader_uuid: "dd79506eab214525bc239ab3b4bb2b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccde81d00a7144e68d82647b6725d676" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd79506eab214525bc239ab3b4bb2b8a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46653 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e445c0d7a5824cb3801a9165ea036347" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33225 } health_report { overall_health: UNKNOWN } } }
W20260809 00:42:15.561214 32423 tablet.cc:2369] T a8c98d23244241689febc6e118b8c5a8 P dd79506eab214525bc239ab3b4bb2b8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:15.569049 32634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.560762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50120 (local address 127.30.104.193:33225)
0809 00:42:15.560872 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.560875 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.560886 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:15.560953 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.560954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.560964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.561025 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:15.561762 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.562269 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.563068 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.563269 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.563335 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.565909 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:15.565922 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:15.565922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:15.565930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:15.567803 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.568181 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.568185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.568185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.568246 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.568478 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.568478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.568478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.568654 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:15.568763 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.568878 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.568948 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:15.595726 32634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.588240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50124 (local address 127.30.104.193:33225)
0809 00:42:15.588330 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.588332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.588420 (+ 88us) server_negotiation.cc:407] Connection header received
0809 00:42:15.588470 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.588471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.588481 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.588546 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:15.589156 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.589642 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.590410 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.590613 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.590681 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.593045 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:15.593046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:15.593046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:15.593054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:15.594411 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.594761 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.594762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.594762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.594822 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.595076 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.595077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.595078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.595189 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:15.595285 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.595548 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.595631 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[31881](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:15.623525 32634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:15.613704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.30.104.193:33225)
0809 00:42:15.613781 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:15.613784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:15.613955 (+ 171us) server_negotiation.cc:407] Connection header received
0809 00:42:15.614157 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:15.614158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:15.614168 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:15.614229 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:15.614886 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.615737 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.617087 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:15.617437 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:15.617521 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:15.620665 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:15.620668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:15.620669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:15.620678 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:15.622323 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.622682 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.622684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.622684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.622745 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:15.623019 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:15.623020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:15.623020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:15.623126 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:15.623202 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:15.623307 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 00:42:15.623373 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 00:42:15.624115 32098 tablet_service.cc:2908] Rejecting scan request for tablet a8c98d23244241689febc6e118b8c5a8: Uninitialized: clean time has not yet been initialized
I20260809 00:42:15.625087 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 32036
I20260809 00:42:15.638341 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 32225
I20260809 00:42:15.648816 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 32428
I20260809 00:42:15.657931 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31901
2026-08-09T00:42:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2225 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[32668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[32668](info): set up 2 sockets
Aug 09 00:42:15 dist-test-slave-c12x krb5kdc[32668](info): commencing operation
krb5kdc: starting...
W20260809 00:42:16.882540 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:16 dist-test-slave-c12x krb5kdc[32668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:17.012825 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37853
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:17.104115 32685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:17.104389 32685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:17.104417 32685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:17.105820 32685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:17.105878 32685 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:42:17.105893 32685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:17.105908 32685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:17.105922 32685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:17.105937 32685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:17.107771 32685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37853
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:17.108103 32685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:17.108386 32685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:17.114701 32695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:17.114802 32694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:17.114677 32699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:17.116537 32685 server_base.cc:1034] running on GCE node
I20260809 00:42:17.116837 32685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:17.117165 32685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:17.118335 32685 hybrid_clock.cc:648] HybridClock initialized: now 1786236137118298 us; error 50 us; skew 500 ppm
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:17.125998 32685 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:17.126544 32685 webserver.cc:460] Webserver started at http://127.30.104.254:44405/ using document root <none> and password file <none>
I20260809 00:42:17.127038 32685 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:17.127318 32685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:17.127629 32685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:17.128953 32685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "93b10f038d554e95b9272b66bc46b4dc"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "c0b3273ea635ed057588176cfd2547bb"
server_key_iv: "48599a27cec82ff05e70aaca5405aba8"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.129441 32685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "93b10f038d554e95b9272b66bc46b4dc"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "c0b3273ea635ed057588176cfd2547bb"
server_key_iv: "48599a27cec82ff05e70aaca5405aba8"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.131095 32685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:17.132107 32706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.132340 32685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:17.132474 32685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "93b10f038d554e95b9272b66bc46b4dc"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "c0b3273ea635ed057588176cfd2547bb"
server_key_iv: "48599a27cec82ff05e70aaca5405aba8"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.132562 32685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:17.160475 32685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:17.162889 32685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:17.163074 32685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:17.168298 32685 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37853
I20260809 00:42:17.168340 328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37853 every 8 connection(s)
I20260809 00:42:17.168730 32685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:17.169546 330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:17.169600 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 32685
I20260809 00:42:17.169715 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:17.169914 31139 external_mini_cluster.cc:1428] Setting key ea990d148c1fc72f5fa23d46d70f6d91
I20260809 00:42:17.173539 330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc: Bootstrap starting.
I20260809 00:42:17.174324 330 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:17.174656 330 log.cc:824] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:17.175591 330 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc: No bootstrap required, opened a new log
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:17.178095 330 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b10f038d554e95b9272b66bc46b4dc" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37853 } }
I20260809 00:42:17.178252 330 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:17.178278 330 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93b10f038d554e95b9272b66bc46b4dc, State: Initialized, Role: FOLLOWER
I20260809 00:42:17.178454 330 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b10f038d554e95b9272b66bc46b4dc" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37853 } }
I20260809 00:42:17.178541 330 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:17.178566 330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:17.178644 330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:17.179582 330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b10f038d554e95b9272b66bc46b4dc" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37853 } }
I20260809 00:42:17.179711 330 leader_election.cc:302] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93b10f038d554e95b9272b66bc46b4dc; no voters:
I20260809 00:42:17.179924 330 leader_election.cc:288] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:17.179991 340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:17.180198 340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [term 1 LEADER]: Becoming Leader. State: Replica: 93b10f038d554e95b9272b66bc46b4dc, State: Running, Role: LEADER
I20260809 00:42:17.180222 330 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:17.180305 340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b10f038d554e95b9272b66bc46b4dc" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37853 } }
I20260809 00:42:17.180693 340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93b10f038d554e95b9272b66bc46b4dc. Latest consensus state: current_term: 1 leader_uuid: "93b10f038d554e95b9272b66bc46b4dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b10f038d554e95b9272b66bc46b4dc" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37853 } } }
I20260809 00:42:17.180845 340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:17.181137 350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:17.181972 350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:17.182289 341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93b10f038d554e95b9272b66bc46b4dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b10f038d554e95b9272b66bc46b4dc" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37853 } } }
I20260809 00:42:17.182775 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.170588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.30.104.254:37853)
0809 00:42:17.170802 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.170813 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.170852 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:42:17.171424 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.171434 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.171545 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.171720 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:17.172351 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.173945 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.174844 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.175272 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.175407 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.178146 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:17.178150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:17.178150 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:17.178161 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:17.180482 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.180988 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.180991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.180992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.181113 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.181374 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.181375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.181375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.181753 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:17.181907 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.182404 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.182558 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
I20260809 00:42:17.183080 341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:17.184170 350 catalog_manager.cc:1347] Generated new cluster ID: ec623bc7087e455e91b955d85eaa33cb
I20260809 00:42:17.184240 350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:17.196686 350 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:17.197253 350 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:17.205209 350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc: Generated new TSK 0
I20260809 00:42:17.205502 350 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:17.238572 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37853
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:17.332804 367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:17.333030 367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:17.333056 367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:17.334471 367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:17.334535 367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:17.334568 367 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:17.334586 367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:17.336493 367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:17.336790 367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:17.337006 367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:17.342410 373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:17.342433 377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:17.342681 367 server_base.cc:1034] running on GCE node
W20260809 00:42:17.342511 374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:17.343075 367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:17.343403 367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:17.344596 367 hybrid_clock.cc:648] HybridClock initialized: now 1786236137344580 us; error 35 us; skew 500 ppm
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:17.347270 367 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:17.347790 367 webserver.cc:460] Webserver started at http://127.30.104.193:34219/ using document root <none> and password file <none>
I20260809 00:42:17.348034 367 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:17.348091 367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:17.348265 367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:17.349386 367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "d0d53ad2ccdd4ed8a325479a70f9f351"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "8c31436fdfc69f6bda3bb498f8617fb3"
server_key_iv: "1be4e013389f22683f2b744d1e9946f5"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.349772 367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "d0d53ad2ccdd4ed8a325479a70f9f351"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "8c31436fdfc69f6bda3bb498f8617fb3"
server_key_iv: "1be4e013389f22683f2b744d1e9946f5"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.351240 367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:17.352240 384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.352451 367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:17.352558 367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "d0d53ad2ccdd4ed8a325479a70f9f351"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "8c31436fdfc69f6bda3bb498f8617fb3"
server_key_iv: "1be4e013389f22683f2b744d1e9946f5"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.352641 367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:17.379190 367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:17.381551 367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:17.381707 367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:17.382382 367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:17.382450 367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.382499 367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:17.382526 367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.389992 367 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:37799
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:17.390044 548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:37799 every 8 connection(s)
I20260809 00:42:17.390381 367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:17.393540 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.382903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:46753 (local address 127.30.104.254:37853)
0809 00:42:17.382965 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.382967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.383201 (+ 234us) server_negotiation.cc:407] Connection header received
0809 00:42:17.383841 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.383843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.383855 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.383915 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:17.384835 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.385712 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.387256 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.387575 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.387656 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.390602 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:17.390605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:17.390605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:17.390615 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:17.392226 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.392672 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.392673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.392674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.392729 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.392958 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.392959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.392959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.393069 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:17.393122 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.393371 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.393443 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:42:17.393688 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.382905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37853 (local address 127.30.104.193:46753)
0809 00:42:17.383075 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.383111 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.383731 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.383998 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.384002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.384401 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:17.384712 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.384719 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.385836 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.385837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.387165 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.387169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.387700 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.387701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.387836 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.388488 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:17.388500 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:17.390499 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:17.392320 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.392328 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.392330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.392593 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.392790 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.392791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.392791 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.392900 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.393127 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:17.393131 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:17.393259 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.393346 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.393481 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260809 00:42:17.394268 549 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37853
I20260809 00:42:17.394381 549 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:17.394574 549 heartbeater.cc:511] Master 127.30.104.254:37853 requested a full tablet report, sending...
I20260809 00:42:17.395145 32728 ts_manager.cc:194] Registered new tserver with Master: d0d53ad2ccdd4ed8a325479a70f9f351 (127.30.104.193:37799)
I20260809 00:42:17.395643 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 367
I20260809 00:42:17.395744 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:17.395910 31139 external_mini_cluster.cc:1428] Setting key a61b6945f5ecb541f0119eb2d24b5599
I20260809 00:42:17.396095 32728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:46753
I20260809 00:42:17.402486 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.395450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:60021 (local address 127.30.104.254:37853)
0809 00:42:17.396309 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.396311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.396318 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:42:17.396336 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.396337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.396345 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.396377 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:17.396960 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.397436 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.398145 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.398460 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.398532 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.399606 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:17.399608 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:17.399608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:17.399617 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:17.401277 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.401685 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.401686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.401687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.401751 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.401956 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.401957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.401957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.402110 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:17.402182 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.402353 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.402418 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":840}
I20260809 00:42:17.402663 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.395356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37853 (local address 127.30.104.193:60021)
0809 00:42:17.396109 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.396122 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.396181 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.396636 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.396637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.396822 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:17.396881 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.396883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.397541 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.397542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.398058 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.398060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.398530 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.398531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.398598 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.399103 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:17.399108 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:17.399513 (+ 405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:17.401380 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.401381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.401382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.401606 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.401824 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.401825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.401825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.401882 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.402188 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:17.402189 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:17.402245 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.402338 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.402502 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:17.426898 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37853
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:17.517488 564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:17.517712 564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:17.517740 564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:17.518875 564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:17.518914 564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:17.518935 564 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:17.518954 564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:17.520434 564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:17.520650 564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:17.520841 564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:17.526181 576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:17.526189 570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:17.526196 572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:17.526561 564 server_base.cc:1034] running on GCE node
I20260809 00:42:17.526818 564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:17.527072 564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:17.528231 564 hybrid_clock.cc:648] HybridClock initialized: now 1786236137528198 us; error 49 us; skew 500 ppm
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:17.530897 564 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:17.531390 564 webserver.cc:460] Webserver started at http://127.30.104.194:36983/ using document root <none> and password file <none>
I20260809 00:42:17.531668 564 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:17.531718 564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:17.531809 564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:17.532838 564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "d73d61096b444efa9e84107691c4870c"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "7ca02b3680d1477623dfe7d1e56cb133"
server_key_iv: "fbca94b0be904786d084aa8e640d7100"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.533147 564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "d73d61096b444efa9e84107691c4870c"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "7ca02b3680d1477623dfe7d1e56cb133"
server_key_iv: "fbca94b0be904786d084aa8e640d7100"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.534514 564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:17.535316 583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.535564 564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:17.535698 564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "d73d61096b444efa9e84107691c4870c"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "7ca02b3680d1477623dfe7d1e56cb133"
server_key_iv: "fbca94b0be904786d084aa8e640d7100"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.535790 564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:17.553676 564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:17.556330 564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:17.556511 564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:17.557224 564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:17.557287 564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.557329 564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:17.557350 564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:17.565604 564 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:46065
I20260809 00:42:17.565765 736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:46065 every 8 connection(s)
I20260809 00:42:17.566028 564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:17.568533 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.557666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:53633 (local address 127.30.104.254:37853)
0809 00:42:17.557740 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.557743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.558053 (+ 310us) server_negotiation.cc:407] Connection header received
0809 00:42:17.558871 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.558873 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.558888 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.558970 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:17.560137 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.560644 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.561717 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.561942 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.562023 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.565372 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:17.565374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:17.565374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:17.565385 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:17.567113 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.567558 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.567559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.567559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.567616 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.567874 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.567875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.567875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.567993 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:17.568059 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.568358 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.568440 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:42:17.568562 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.557667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37853 (local address 127.30.104.194:53633)
0809 00:42:17.557878 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.557940 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.558745 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.559129 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.559134 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.559665 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:17.560010 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.560018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.560754 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.560755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.561611 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.561614 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.562055 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.562056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.562200 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.562745 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:17.562762 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:17.565233 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:17.567190 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.567194 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.567196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.567419 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.567692 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.567692 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.567693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.567800 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.568058 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:17.568063 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:17.568186 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.568268 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.568397 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260809 00:42:17.569316 737 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37853
I20260809 00:42:17.569412 737 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:17.569604 737 heartbeater.cc:511] Master 127.30.104.254:37853 requested a full tablet report, sending...
I20260809 00:42:17.570011 32728 ts_manager.cc:194] Registered new tserver with Master: d73d61096b444efa9e84107691c4870c (127.30.104.194:46065)
I20260809 00:42:17.570608 32728 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:53633
I20260809 00:42:17.573752 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 564
I20260809 00:42:17.573853 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:17.574059 31139 external_mini_cluster.cc:1428] Setting key 568a011caafb6d5c09f5cdfbcf469b19
I20260809 00:42:17.576949 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.570524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:34593 (local address 127.30.104.254:37853)
0809 00:42:17.570591 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.570593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.570788 (+ 195us) server_negotiation.cc:407] Connection header received
0809 00:42:17.570814 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.570815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.570827 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.570910 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:17.571369 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.571912 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.572606 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.572787 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.572846 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.574043 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:17.574044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:17.574045 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:17.574051 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:17.575646 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.576058 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.576059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.576059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.576127 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.576335 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.576335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.576335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.576479 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:17.576571 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.576769 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.576850 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:42:17.576963 745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.570454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37853 (local address 127.30.104.194:34593)
0809 00:42:17.570598 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.570608 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.570692 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.570936 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.570937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.571207 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:17.571274 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.571277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.572007 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.572007 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.572518 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.572520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.572885 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.572886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.572965 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.573404 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:17.573408 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:17.573878 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:17.575725 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.575726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.575727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.575981 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.576193 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.576194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.576194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.576267 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.576562 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:17.576563 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:17.576647 (+ 84us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.576732 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.576852 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:17.602595 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37853
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:17.694237 748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:17.694578 748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:17.694625 748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:17.695931 748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:17.696000 748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:17.696034 748 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:17.696076 748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:17.697649 748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:37853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:17.697932 748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:17.698221 748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:17.703760 756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:17.703809 755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:17.703788 748 server_base.cc:1034] running on GCE node
W20260809 00:42:17.703760 758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:17.704136 748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:17.704409 748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:17.705569 748 hybrid_clock.cc:648] HybridClock initialized: now 1786236137705554 us; error 29 us; skew 500 ppm
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:17.708425 748 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:17.708964 748 webserver.cc:460] Webserver started at http://127.30.104.195:43917/ using document root <none> and password file <none>
I20260809 00:42:17.709247 748 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:17.709322 748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:17.709482 748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:17.710628 748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "b346f870505d4b17a05448ffb66a1929"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "6b4a27b955a5472d4b9768c27e2e9862"
server_key_iv: "5980740ae042ebec4d9bafeb7d44882d"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.711014 748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "b346f870505d4b17a05448ffb66a1929"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "6b4a27b955a5472d4b9768c27e2e9862"
server_key_iv: "5980740ae042ebec4d9bafeb7d44882d"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.712520 748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:17.713373 767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.713626 748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:17.713724 748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "b346f870505d4b17a05448ffb66a1929"
format_stamp: "Formatted at 2026-08-09 00:42:17 on dist-test-slave-c12x"
server_key: "6b4a27b955a5472d4b9768c27e2e9862"
server_key_iv: "5980740ae042ebec4d9bafeb7d44882d"
server_key_version: "encryptionkey@0"
I20260809 00:42:17.713783 748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:17.726548 748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:17.729018 748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:17.729164 748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:17.729661 748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:17.729740 748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.729815 748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:17.729846 748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:17.737973 748 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:35701
I20260809 00:42:17.738101 920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:35701 every 8 connection(s)
I20260809 00:42:17.738431 748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:17.738821 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 748
I20260809 00:42:17.738919 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:17.739099 31139 external_mini_cluster.cc:1428] Setting key 41600d937f8f6d0761bd42e85404b248
I20260809 00:42:17.742074 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.730305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:38133 (local address 127.30.104.254:37853)
0809 00:42:17.730383 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.730385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.730630 (+ 245us) server_negotiation.cc:407] Connection header received
0809 00:42:17.731261 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.731263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.731275 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.731344 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:17.732582 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.733330 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.735103 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.735504 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.735590 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.738608 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:17.738610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:17.738611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:17.738622 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:17.740493 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.740996 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.740997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.740997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.741063 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.741364 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.741365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.741365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.741490 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:17.741546 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.741926 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.742007 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:42:17.742281 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.730302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37853 (local address 127.30.104.195:38133)
0809 00:42:17.730495 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.730532 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.731129 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.731610 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.731617 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.732094 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:17.732442 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.732450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.733445 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.733446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.734972 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.734977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.735627 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.735628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.735805 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.736429 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:17.736441 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:17.738452 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:17.740607 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.740613 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.740615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.740890 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.741165 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.741166 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.741166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.741290 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.741553 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:17.741558 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:17.741769 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.741893 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.742043 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
I20260809 00:42:17.743278 923 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37853
I20260809 00:42:17.743414 923 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:17.743643 923 heartbeater.cc:511] Master 127.30.104.254:37853 requested a full tablet report, sending...
I20260809 00:42:17.744149 32726 ts_manager.cc:194] Registered new tserver with Master: b346f870505d4b17a05448ffb66a1929 (127.30.104.195:35701)
I20260809 00:42:17.744639 32726 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:38133
I20260809 00:42:17.750921 933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.744620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37853 (local address 127.30.104.195:39483)
0809 00:42:17.744787 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.744798 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.744861 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.745013 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.745014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.745268 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:17.745337 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.745340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.746035 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.746036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.746567 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.746569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.746952 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.746953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.747016 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.747524 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:17.747530 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:17.747868 (+ 338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:17.749528 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.749529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.749529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.749726 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.750105 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:17.750106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:17.750106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:17.750167 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:17.750527 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:17.750528 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:17.750588 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.750657 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.750787 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":91,"threads_started":1}
I20260809 00:42:17.750981 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.744697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:39483 (local address 127.30.104.254:37853)
0809 00:42:17.744745 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.744747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.744865 (+ 118us) server_negotiation.cc:407] Connection header received
0809 00:42:17.744913 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.744914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.744925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.745011 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:17.745467 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.745933 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.746659 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.746869 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.746935 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.747954 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:17.747955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:17.747955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:17.747962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:17.749437 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.749871 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.749874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.749875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.750000 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.750268 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.750269 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.750269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.750403 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:17.750499 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.750748 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.750853 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:42:17.751664 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:42:17 dist-test-slave-c12x krb5kdc[32668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:17.778924 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.770680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59202 (local address 127.30.104.254:37853)
0809 00:42:17.770747 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.770749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.770759 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:17.770837 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.770837 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.770847 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.770901 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:17.771583 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.772109 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.772871 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.773071 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.773129 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.776110 (+ 2981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:17.776112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:17.776112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:17.776120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:17.777607 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.778027 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.778028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.778028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.778088 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:17.778330 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:17.778330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:17.778330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:17.778454 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:17.778532 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.778768 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.778844 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:17.781080 32726 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:17.782892 32726 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:17.789959 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.785609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42224 (local address 127.30.104.193:37799)
0809 00:42:17.786510 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.786513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.786521 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:17.786551 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.786553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.786564 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.786623 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:17.786784 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.787584 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.788956 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.789455 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.789554 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.789615 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.789800 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.789843 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":870,"thread_start_us":116,"threads_started":1}
I20260809 00:42:17.789958 953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.785476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:37799 (local address 127.0.0.1:42224)
0809 00:42:17.785826 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.785869 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.785938 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.786653 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.786654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.786657 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:17.786711 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.786712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.787656 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.787657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.788853 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.788856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.789551 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.789552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.789661 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.789676 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.789781 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.789851 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":28,"thread_start_us":101,"threads_started":1}
I20260809 00:42:17.789964 945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.785450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:35701 (local address 127.0.0.1:52728)
0809 00:42:17.785825 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.785869 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.785938 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.786252 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.786255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.786257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:17.786335 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.786338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.787358 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.787359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.788570 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.788573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.789662 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.789663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.789723 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.789727 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.789771 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.789844 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":22,"thread_start_us":81,"threads_started":1}
I20260809 00:42:17.790179 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.785617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52728 (local address 127.30.104.195:35701)
0809 00:42:17.786090 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.786094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.786102 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:17.786135 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.786137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.786149 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.786234 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:17.786423 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.787266 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.788663 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.789503 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.789580 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.789628 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.790006 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.790055 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":99,"threads_started":1}
I20260809 00:42:17.790849 454 tablet_service.cc:1467] Processing CreateTablet for tablet b32a2aacfcd441f8936d7790790620d9 (DEFAULT_TABLE table=test-table [id=96c9b9af26934de083dd2702b4f184d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:17.790967 837 tablet_service.cc:1467] Processing CreateTablet for tablet b32a2aacfcd441f8936d7790790620d9 (DEFAULT_TABLE table=test-table [id=96c9b9af26934de083dd2702b4f184d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:17.791281 454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b32a2aacfcd441f8936d7790790620d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:17.791324 837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b32a2aacfcd441f8936d7790790620d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:17.791875 946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.785702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:46065 (local address 127.0.0.1:60748)
0809 00:42:17.786323 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.786333 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.786388 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.787521 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.787522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.787525 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:17.787597 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.787598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.788879 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.788880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.789956 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.789958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.791594 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.791595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.791674 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.791679 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.791756 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.791799 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":80,"threads_started":1}
I20260809 00:42:17.791929 956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.785844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.30.104.194:46065)
0809 00:42:17.786008 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.786013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.787345 (+ 1332us) server_negotiation.cc:407] Connection header received
0809 00:42:17.787388 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.787390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.787403 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.787522 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:17.787702 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.788800 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.790698 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.791508 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.791588 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.791642 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.791783 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.791830 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260809 00:42:17.792858 643 tablet_service.cc:1467] Processing CreateTablet for tablet b32a2aacfcd441f8936d7790790620d9 (DEFAULT_TABLE table=test-table [id=96c9b9af26934de083dd2702b4f184d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:17.793226 643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b32a2aacfcd441f8936d7790790620d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:17.794747 957 tablet_bootstrap.cc:492] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Bootstrap starting.
I20260809 00:42:17.795261 958 tablet_bootstrap.cc:492] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: Bootstrap starting.
I20260809 00:42:17.795641 957 tablet_bootstrap.cc:654] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:17.796061 957 log.cc:824] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:17.796309 958 tablet_bootstrap.cc:654] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:17.796627 958 log.cc:824] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:17.797021 957 tablet_bootstrap.cc:492] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: No bootstrap required, opened a new log
I20260809 00:42:17.797114 957 ts_tablet_manager.cc:1397] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:42:17.797454 958 tablet_bootstrap.cc:492] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: No bootstrap required, opened a new log
I20260809 00:42:17.797524 958 ts_tablet_manager.cc:1397] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:17.799355 957 raft_consensus.cc:348] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.799600 957 raft_consensus.cc:374] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:17.799642 957 raft_consensus.cc:729] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d53ad2ccdd4ed8a325479a70f9f351, State: Initialized, Role: FOLLOWER
I20260809 00:42:17.799768 957 consensus_queue.cc:260] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.799703 958 raft_consensus.cc:348] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.799850 958 raft_consensus.cc:374] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:17.799896 958 raft_consensus.cc:729] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b346f870505d4b17a05448ffb66a1929, State: Initialized, Role: FOLLOWER
I20260809 00:42:17.800051 957 ts_tablet_manager.cc:1428] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 00:42:17.800118 549 heartbeater.cc:503] Master 127.30.104.254:37853 was elected leader, sending a full tablet report...
I20260809 00:42:17.800027 958 consensus_queue.cc:260] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.800586 958 ts_tablet_manager.cc:1428] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:42:17.800626 923 heartbeater.cc:503] Master 127.30.104.254:37853 was elected leader, sending a full tablet report...
I20260809 00:42:17.800776 961 tablet_bootstrap.cc:492] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Bootstrap starting.
I20260809 00:42:17.801685 961 tablet_bootstrap.cc:654] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:17.801975 961 log.cc:824] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:17.802677 961 tablet_bootstrap.cc:492] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: No bootstrap required, opened a new log
I20260809 00:42:17.802735 961 ts_tablet_manager.cc:1397] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:17.804220 961 raft_consensus.cc:348] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.804354 961 raft_consensus.cc:374] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:17.804375 961 raft_consensus.cc:729] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d73d61096b444efa9e84107691c4870c, State: Initialized, Role: FOLLOWER
I20260809 00:42:17.804497 961 consensus_queue.cc:260] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.804728 961 ts_tablet_manager.cc:1428] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:17.804836 737 heartbeater.cc:503] Master 127.30.104.254:37853 was elected leader, sending a full tablet report...
W20260809 00:42:17.816671 738 tablet.cc:2369] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:42:17.891256 550 tablet.cc:2369] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:42:17.912252 963 raft_consensus.cc:482] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:17.912382 963 raft_consensus.cc:504] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.912967 963 leader_election.cc:288] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0d53ad2ccdd4ed8a325479a70f9f351 (127.30.104.193:37799), d73d61096b444efa9e84107691c4870c (127.30.104.194:46065)
I20260809 00:42:17.916180 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.913319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:59293 (local address 127.30.104.193:37799)
0809 00:42:17.913387 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.913389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.913416 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:42:17.913454 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.913454 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.913467 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.913548 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:17.913728 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.914217 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.915308 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.915757 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.915865 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.915875 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.916065 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.916108 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:42:17.916180 778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.913249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:37799 (local address 127.30.104.195:59293)
0809 00:42:17.913316 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.913330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.913384 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.913567 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.913568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.913570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:17.913648 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.913650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.914322 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.914323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.915182 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.915186 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.915865 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.915866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.915960 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.915964 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.916057 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.916103 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 00:42:17.916568 967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.913431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:46065 (local address 127.30.104.195:47491)
0809 00:42:17.913653 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:17.913662 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:17.913739 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:17.913924 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:17.913924 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:17.913926 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:17.914024 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.914026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.914750 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.914751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.915594 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:17.915596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.916244 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.916246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:17.916340 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.916360 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 00:42:17.916431 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:42:17.916473 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":127,"threads_started":1}
I20260809 00:42:17.916553 956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.913578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:47491 (local address 127.30.104.194:46065)
0809 00:42:17.913637 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.913640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.913727 (+ 87us) server_negotiation.cc:407] Connection header received
0809 00:42:17.913810 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.913811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.913828 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.913916 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:17.914099 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.914656 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.915689 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.916085 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.916192 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.916201 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.916439 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.916482 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:42:17.916554 479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b32a2aacfcd441f8936d7790790620d9" candidate_uuid: "b346f870505d4b17a05448ffb66a1929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" is_pre_election: true
I20260809 00:42:17.916739 479 raft_consensus.cc:2454] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b346f870505d4b17a05448ffb66a1929 in term 0.
I20260809 00:42:17.917016 771 leader_election.cc:302] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b346f870505d4b17a05448ffb66a1929, d0d53ad2ccdd4ed8a325479a70f9f351; no voters:
I20260809 00:42:17.917158 963 raft_consensus.cc:2781] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:17.917122 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b32a2aacfcd441f8936d7790790620d9" candidate_uuid: "b346f870505d4b17a05448ffb66a1929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d73d61096b444efa9e84107691c4870c" is_pre_election: true
I20260809 00:42:17.917220 963 raft_consensus.cc:482] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:17.917238 963 raft_consensus.cc:3037] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:17.917303 670 raft_consensus.cc:2454] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b346f870505d4b17a05448ffb66a1929 in term 0.
I20260809 00:42:17.918316 963 raft_consensus.cc:504] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.918462 963 leader_election.cc:288] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [CANDIDATE]: Term 1 election: Requested vote from peers d0d53ad2ccdd4ed8a325479a70f9f351 (127.30.104.193:37799), d73d61096b444efa9e84107691c4870c (127.30.104.194:46065)
I20260809 00:42:17.918716 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b32a2aacfcd441f8936d7790790620d9" candidate_uuid: "b346f870505d4b17a05448ffb66a1929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d73d61096b444efa9e84107691c4870c"
I20260809 00:42:17.918690 479 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b32a2aacfcd441f8936d7790790620d9" candidate_uuid: "b346f870505d4b17a05448ffb66a1929" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351"
I20260809 00:42:17.918788 670 raft_consensus.cc:3037] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:17.918814 479 raft_consensus.cc:3037] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:17.919556 670 raft_consensus.cc:2454] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b346f870505d4b17a05448ffb66a1929 in term 1.
I20260809 00:42:17.919719 771 leader_election.cc:302] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b346f870505d4b17a05448ffb66a1929, d73d61096b444efa9e84107691c4870c; no voters:
I20260809 00:42:17.919734 479 raft_consensus.cc:2454] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b346f870505d4b17a05448ffb66a1929 in term 1.
I20260809 00:42:17.919826 963 raft_consensus.cc:2781] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:17.920015 963 raft_consensus.cc:686] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 1 LEADER]: Becoming Leader. State: Replica: b346f870505d4b17a05448ffb66a1929, State: Running, Role: LEADER
I20260809 00:42:17.920142 963 consensus_queue.cc:237] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } }
I20260809 00:42:17.921150 32726 catalog_manager.cc:5495] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 reported cstate change: term changed from 0 to 1, leader changed from <none> to b346f870505d4b17a05448ffb66a1929 (127.30.104.195). New cstate: current_term: 1 leader_uuid: "b346f870505d4b17a05448ffb66a1929" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:17.940989 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:17.938416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.30.104.195:35701)
0809 00:42:17.938468 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:17.938470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:17.938478 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:17.938513 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:17.938513 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:17.938529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:17.938596 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:17.938752 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.939307 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.940170 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:17.940359 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:17.940417 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:17.940602 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:17.940700 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:17.940762 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:17.940862 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 00:42:17.940884 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:17.943007 479 raft_consensus.cc:1261] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Refusing update from remote peer b346f870505d4b17a05448ffb66a1929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:17.943028 670 raft_consensus.cc:1261] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Refusing update from remote peer b346f870505d4b17a05448ffb66a1929: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:17.943454 963 consensus_queue.cc:1035] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:17.943634 969 consensus_queue.cc:1035] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:17.946647 974 mvcc.cc:205] Tried to move back new op lower bound from 7316423221011554304 to 7316423220921688064. Current Snapshot: MvccSnapshot[applied={T|T < 7316423221011554304}]
I20260809 00:42:17.946698 973 mvcc.cc:205] Tried to move back new op lower bound from 7316423221011554304 to 7316423220921688064. Current Snapshot: MvccSnapshot[applied={T|T < 7316423221011554304}]
I20260809 00:42:17.950172 32728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:17.950294 32728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:17.951143 32728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc: Sending DeleteTablet for 3 replicas of tablet b32a2aacfcd441f8936d7790790620d9
I20260809 00:42:17.951526 837 tablet_service.cc:1514] Processing DeleteTablet for tablet b32a2aacfcd441f8936d7790790620d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:17 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:52728
I20260809 00:42:17.951653 454 tablet_service.cc:1514] Processing DeleteTablet for tablet b32a2aacfcd441f8936d7790790620d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:17 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:42224
I20260809 00:42:17.951668 643 tablet_service.cc:1514] Processing DeleteTablet for tablet b32a2aacfcd441f8936d7790790620d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:17 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:60748
I20260809 00:42:17.951772 985 tablet_replica.cc:330] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: stopping tablet replica
I20260809 00:42:17.951869 985 raft_consensus.cc:2229] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:17.951997 985 raft_consensus.cc:2258] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:17.952263 986 tablet_replica.cc:330] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: stopping tablet replica
I20260809 00:42:17.952252 32728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59202:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 00:42:17.952370 986 raft_consensus.cc:2229] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260809 00:42:17.952411 32728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:17.952469 986 raft_consensus.cc:2258] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:17.952486 985 ts_tablet_manager.cc:1905] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:17.952859 987 tablet_replica.cc:330] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: stopping tablet replica
I20260809 00:42:17.952963 987 raft_consensus.cc:2229] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:17.953012 986 ts_tablet_manager.cc:1905] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:17.953080 987 raft_consensus.cc:2258] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:17.953526 987 ts_tablet_manager.cc:1905] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:17.954316 985 ts_tablet_manager.cc:1918] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:17.954388 985 log.cc:1196] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/b32a2aacfcd441f8936d7790790620d9
I20260809 00:42:17.954686 985 ts_tablet_manager.cc:1939] T b32a2aacfcd441f8936d7790790620d9 P b346f870505d4b17a05448ffb66a1929: Deleting consensus metadata
I20260809 00:42:17.954720 454 tablet_service.cc:1467] Processing CreateTablet for tablet decfa58805a241a49ceb7b6e45f147d8 (DEFAULT_TABLE table=test-table [id=81e2eecf19264b57bbce91321f42da37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:17.954756 986 ts_tablet_manager.cc:1918] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:17.954810 986 log.cc:1196] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/b32a2aacfcd441f8936d7790790620d9
I20260809 00:42:17.954830 454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decfa58805a241a49ceb7b6e45f147d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:17.955076 986 ts_tablet_manager.cc:1939] T b32a2aacfcd441f8936d7790790620d9 P d0d53ad2ccdd4ed8a325479a70f9f351: Deleting consensus metadata
I20260809 00:42:17.955205 987 ts_tablet_manager.cc:1918] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:17.955278 987 log.cc:1196] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/b32a2aacfcd441f8936d7790790620d9
I20260809 00:42:17.955559 32709 catalog_manager.cc:4841] TS d0d53ad2ccdd4ed8a325479a70f9f351 (127.30.104.193:37799): tablet b32a2aacfcd441f8936d7790790620d9 (table test-table [id=96c9b9af26934de083dd2702b4f184d0]) successfully deleted
I20260809 00:42:17.955613 987 ts_tablet_manager.cc:1939] T b32a2aacfcd441f8936d7790790620d9 P d73d61096b444efa9e84107691c4870c: Deleting consensus metadata
I20260809 00:42:17.956010 32709 catalog_manager.cc:4841] TS d73d61096b444efa9e84107691c4870c (127.30.104.194:46065): tablet b32a2aacfcd441f8936d7790790620d9 (table test-table [id=96c9b9af26934de083dd2702b4f184d0]) successfully deleted
I20260809 00:42:17.956099 957 tablet_bootstrap.cc:492] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: Bootstrap starting.
I20260809 00:42:17.956192 837 tablet_service.cc:1467] Processing CreateTablet for tablet decfa58805a241a49ceb7b6e45f147d8 (DEFAULT_TABLE table=test-table [id=81e2eecf19264b57bbce91321f42da37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:17.956308 837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decfa58805a241a49ceb7b6e45f147d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:17.956470 957 tablet_bootstrap.cc:654] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:17.957082 957 tablet_bootstrap.cc:492] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: No bootstrap required, opened a new log
I20260809 00:42:17.957114 957 ts_tablet_manager.cc:1397] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:17.957468 957 raft_consensus.cc:348] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:17.957558 957 raft_consensus.cc:374] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:17.957580 957 raft_consensus.cc:729] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d53ad2ccdd4ed8a325479a70f9f351, State: Initialized, Role: FOLLOWER
I20260809 00:42:17.957582 958 tablet_bootstrap.cc:492] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: Bootstrap starting.
I20260809 00:42:17.957633 957 consensus_queue.cc:260] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:17.957660 32710 catalog_manager.cc:4841] TS b346f870505d4b17a05448ffb66a1929 (127.30.104.195:35701): tablet b32a2aacfcd441f8936d7790790620d9 (table test-table [id=96c9b9af26934de083dd2702b4f184d0]) successfully deleted
I20260809 00:42:17.957736 957 ts_tablet_manager.cc:1428] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:17.957850 643 tablet_service.cc:1467] Processing CreateTablet for tablet decfa58805a241a49ceb7b6e45f147d8 (DEFAULT_TABLE table=test-table [id=81e2eecf19264b57bbce91321f42da37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:17.957957 643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet decfa58805a241a49ceb7b6e45f147d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:17.958027 958 tablet_bootstrap.cc:654] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:17.958678 958 tablet_bootstrap.cc:492] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: No bootstrap required, opened a new log
I20260809 00:42:17.958712 958 ts_tablet_manager.cc:1397] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:17.958828 958 raft_consensus.cc:348] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:17.958879 958 raft_consensus.cc:374] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:17.958892 958 raft_consensus.cc:729] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b346f870505d4b17a05448ffb66a1929, State: Initialized, Role: FOLLOWER
I20260809 00:42:17.958959 958 consensus_queue.cc:260] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:17.959053 958 ts_tablet_manager.cc:1428] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:17.959167 961 tablet_bootstrap.cc:492] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: Bootstrap starting.
I20260809 00:42:17.959650 961 tablet_bootstrap.cc:654] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:17.960296 961 tablet_bootstrap.cc:492] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: No bootstrap required, opened a new log
I20260809 00:42:17.960338 961 ts_tablet_manager.cc:1397] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:17.960667 961 raft_consensus.cc:348] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:17.960733 961 raft_consensus.cc:374] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:17.960747 961 raft_consensus.cc:729] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d73d61096b444efa9e84107691c4870c, State: Initialized, Role: FOLLOWER
I20260809 00:42:17.960811 961 consensus_queue.cc:260] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:17.960985 961 ts_tablet_manager.cc:1428] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
W20260809 00:42:17.989796 925 tablet.cc:2369] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:42:18.009261 962 raft_consensus.cc:482] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:18.009368 962 raft_consensus.cc:504] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.009749 962 leader_election.cc:288] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d73d61096b444efa9e84107691c4870c (127.30.104.194:46065), b346f870505d4b17a05448ffb66a1929 (127.30.104.195:35701)
I20260809 00:42:18.013554 956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.010013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:43047 (local address 127.30.104.194:46065)
0809 00:42:18.010069 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:18.010072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:18.010209 (+ 137us) server_negotiation.cc:407] Connection header received
0809 00:42:18.010312 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:18.010314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:18.010323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:18.010353 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:18.010671 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.011422 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.012726 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.013098 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.013192 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.013204 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:18.013427 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 00:42:18.013478 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:42:18.013581 994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.009970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:46065 (local address 127.30.104.193:43047)
0809 00:42:18.010152 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:18.010162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:18.010233 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:18.010428 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:18.010429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:18.010431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:18.010556 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.010559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.011541 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.011542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.012605 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.012607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.013212 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.013214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.013289 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.013293 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:18.013400 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 00:42:18.013450 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":37,"thread_start_us":122,"threads_started":1}
I20260809 00:42:18.013638 947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.010145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:60513 (local address 127.30.104.195:35701)
0809 00:42:18.010193 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:18.010195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:18.010205 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:18.010303 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:18.010305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:18.010318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:18.010353 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:18.010662 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.011136 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.012712 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.013255 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.013348 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.013358 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:18.013521 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 00:42:18.013564 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:42:18.013875 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.009966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:35701 (local address 127.30.104.193:60513)
0809 00:42:18.010124 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:18.010137 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:18.010203 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:18.010428 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:18.010429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:18.010432 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:18.010570 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.010572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.011238 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.011239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.012589 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.012592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.013355 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.013356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.013433 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.013435 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:42:18.013720 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 00:42:18.013774 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":53,"thread_start_us":63,"threads_started":1}
I20260809 00:42:18.013825 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decfa58805a241a49ceb7b6e45f147d8" candidate_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d73d61096b444efa9e84107691c4870c" is_pre_election: true
I20260809 00:42:18.013926 670 raft_consensus.cc:2454] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0d53ad2ccdd4ed8a325479a70f9f351 in term 0.
I20260809 00:42:18.014181 388 leader_election.cc:302] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0d53ad2ccdd4ed8a325479a70f9f351, d73d61096b444efa9e84107691c4870c; no voters:
I20260809 00:42:18.014190 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decfa58805a241a49ceb7b6e45f147d8" candidate_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b346f870505d4b17a05448ffb66a1929" is_pre_election: true
I20260809 00:42:18.014308 962 raft_consensus.cc:2781] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:18.014313 861 raft_consensus.cc:2454] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0d53ad2ccdd4ed8a325479a70f9f351 in term 0.
I20260809 00:42:18.014338 962 raft_consensus.cc:482] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:18.014359 962 raft_consensus.cc:3037] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:18.015156 962 raft_consensus.cc:504] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.015331 962 leader_election.cc:288] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [CANDIDATE]: Term 1 election: Requested vote from peers d73d61096b444efa9e84107691c4870c (127.30.104.194:46065), b346f870505d4b17a05448ffb66a1929 (127.30.104.195:35701)
I20260809 00:42:18.015538 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decfa58805a241a49ceb7b6e45f147d8" candidate_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d73d61096b444efa9e84107691c4870c"
I20260809 00:42:18.015636 670 raft_consensus.cc:3037] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:18.015626 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "decfa58805a241a49ceb7b6e45f147d8" candidate_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b346f870505d4b17a05448ffb66a1929"
I20260809 00:42:18.015751 861 raft_consensus.cc:3037] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:18.016397 861 raft_consensus.cc:2454] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0d53ad2ccdd4ed8a325479a70f9f351 in term 1.
I20260809 00:42:18.016494 670 raft_consensus.cc:2454] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0d53ad2ccdd4ed8a325479a70f9f351 in term 1.
I20260809 00:42:18.016604 389 leader_election.cc:302] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b346f870505d4b17a05448ffb66a1929, d0d53ad2ccdd4ed8a325479a70f9f351; no voters:
I20260809 00:42:18.016698 962 raft_consensus.cc:2781] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:18.016933 962 raft_consensus.cc:686] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 LEADER]: Becoming Leader. State: Replica: d0d53ad2ccdd4ed8a325479a70f9f351, State: Running, Role: LEADER
I20260809 00:42:18.017020 962 consensus_queue.cc:237] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.017851 32728 catalog_manager.cc:5495] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0d53ad2ccdd4ed8a325479a70f9f351 (127.30.104.193). New cstate: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:42:18 dist-test-slave-c12x krb5kdc[32668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:18.072866 1002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.064309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37853 (local address 127.0.0.1:59214)
0809 00:42:18.064521 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:18.064537 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:18.064598 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:18.064781 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:18.064782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:18.065102 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:18.065156 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.065158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.066203 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.066204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.066747 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.066748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.067294 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.067295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.067342 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.067950 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:18.067954 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:18.069667 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:18.071693 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:18.071702 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:18.071704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:18.071921 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:18.072127 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:18.072128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:18.072128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:18.072191 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:18.072517 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:18.072525 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:18.072580 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 00:42:18.072676 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 00:42:18.072786 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
I20260809 00:42:18.072973 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.064438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59214 (local address 127.30.104.254:37853)
0809 00:42:18.064516 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:18.064519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:18.064605 (+ 86us) server_negotiation.cc:407] Connection header received
0809 00:42:18.064643 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:18.064645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:18.064656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:18.064745 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:18.065269 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.066102 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.066855 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.067213 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.067294 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.069759 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:18.069761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:18.069761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:18.069769 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:18.071598 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:18.071997 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:18.071998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:18.071999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:18.072059 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:18.072253 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:18.072254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:18.072254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:18.072394 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:18.072459 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:18.072716 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 00:42:18.072849 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":47}
I20260809 00:42:18.075330 32728 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:59214:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:42:18.075662 32728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:18.077268 837 tablet_service.cc:1467] Processing CreateTablet for tablet cee871e7eb4942b7a56591a6f10641c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57115498337544998e6eba0c5b574b18]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:18.077275 643 tablet_service.cc:1467] Processing CreateTablet for tablet cee871e7eb4942b7a56591a6f10641c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57115498337544998e6eba0c5b574b18]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:18.077466 837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cee871e7eb4942b7a56591a6f10641c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:18.077466 643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cee871e7eb4942b7a56591a6f10641c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:18.078848 961 tablet_bootstrap.cc:492] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: Bootstrap starting.
I20260809 00:42:18.078896 958 tablet_bootstrap.cc:492] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: Bootstrap starting.
I20260809 00:42:18.079221 958 tablet_bootstrap.cc:654] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:18.079226 961 tablet_bootstrap.cc:654] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:18.079697 454 tablet_service.cc:1467] Processing CreateTablet for tablet cee871e7eb4942b7a56591a6f10641c0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=57115498337544998e6eba0c5b574b18]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:18.079878 454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cee871e7eb4942b7a56591a6f10641c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:18.080040 961 tablet_bootstrap.cc:492] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: No bootstrap required, opened a new log
I20260809 00:42:18.080112 961 ts_tablet_manager.cc:1397] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:18.080286 961 raft_consensus.cc:348] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.080371 961 raft_consensus.cc:374] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:18.080396 961 raft_consensus.cc:729] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d73d61096b444efa9e84107691c4870c, State: Initialized, Role: FOLLOWER
I20260809 00:42:18.080454 961 consensus_queue.cc:260] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.080603 961 ts_tablet_manager.cc:1428] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:18.080668 966 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.080778 966 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:18.081418 957 tablet_bootstrap.cc:492] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: Bootstrap starting.
I20260809 00:42:18.081668 958 tablet_bootstrap.cc:492] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: No bootstrap required, opened a new log
I20260809 00:42:18.081719 958 ts_tablet_manager.cc:1397] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:42:18.081812 958 raft_consensus.cc:348] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.081862 958 raft_consensus.cc:374] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:18.081878 958 raft_consensus.cc:729] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b346f870505d4b17a05448ffb66a1929, State: Initialized, Role: FOLLOWER
I20260809 00:42:18.081913 958 consensus_queue.cc:260] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.081967 957 tablet_bootstrap.cc:654] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:18.082010 958 ts_tablet_manager.cc:1428] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:18.082046 980 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.082113 980 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:18.082860 957 tablet_bootstrap.cc:492] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: No bootstrap required, opened a new log
I20260809 00:42:18.082929 957 ts_tablet_manager.cc:1397] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 00:42:18.083087 957 raft_consensus.cc:348] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.083165 957 raft_consensus.cc:374] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:18.083191 957 raft_consensus.cc:729] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0d53ad2ccdd4ed8a325479a70f9f351, State: Initialized, Role: FOLLOWER
I20260809 00:42:18.083242 957 consensus_queue.cc:260] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.083339 957 ts_tablet_manager.cc:1428] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:18.083423 962 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.083565 962 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:18.354048 962 raft_consensus.cc:482] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:18.354210 962 raft_consensus.cc:504] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.354456 962 leader_election.cc:288] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d73d61096b444efa9e84107691c4870c (127.30.104.194:46065), b346f870505d4b17a05448ffb66a1929 (127.30.104.195:35701)
I20260809 00:42:18.354730 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cee871e7eb4942b7a56591a6f10641c0" candidate_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d73d61096b444efa9e84107691c4870c" is_pre_election: true
I20260809 00:42:18.354825 670 raft_consensus.cc:2454] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0d53ad2ccdd4ed8a325479a70f9f351 in term 0.
I20260809 00:42:18.354756 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cee871e7eb4942b7a56591a6f10641c0" candidate_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b346f870505d4b17a05448ffb66a1929" is_pre_election: true
I20260809 00:42:18.354899 861 raft_consensus.cc:2454] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0d53ad2ccdd4ed8a325479a70f9f351 in term 0.
I20260809 00:42:18.355019 388 leader_election.cc:302] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0d53ad2ccdd4ed8a325479a70f9f351, d73d61096b444efa9e84107691c4870c; no voters:
I20260809 00:42:18.355149 962 raft_consensus.cc:2781] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:18.355221 962 raft_consensus.cc:482] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:18.355255 962 raft_consensus.cc:3037] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:18.356072 962 raft_consensus.cc:504] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.356213 962 leader_election.cc:288] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [CANDIDATE]: Term 1 election: Requested vote from peers d73d61096b444efa9e84107691c4870c (127.30.104.194:46065), b346f870505d4b17a05448ffb66a1929 (127.30.104.195:35701)
I20260809 00:42:18.356449 861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cee871e7eb4942b7a56591a6f10641c0" candidate_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b346f870505d4b17a05448ffb66a1929"
I20260809 00:42:18.356420 670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cee871e7eb4942b7a56591a6f10641c0" candidate_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d73d61096b444efa9e84107691c4870c"
I20260809 00:42:18.356521 861 raft_consensus.cc:3037] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:18.356545 670 raft_consensus.cc:3037] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:18.357244 861 raft_consensus.cc:2454] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0d53ad2ccdd4ed8a325479a70f9f351 in term 1.
I20260809 00:42:18.357369 670 raft_consensus.cc:2454] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0d53ad2ccdd4ed8a325479a70f9f351 in term 1.
I20260809 00:42:18.357436 389 leader_election.cc:302] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b346f870505d4b17a05448ffb66a1929, d0d53ad2ccdd4ed8a325479a70f9f351; no voters:
I20260809 00:42:18.357545 962 raft_consensus.cc:2781] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:18.357609 962 raft_consensus.cc:686] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 LEADER]: Becoming Leader. State: Replica: d0d53ad2ccdd4ed8a325479a70f9f351, State: Running, Role: LEADER
I20260809 00:42:18.357657 962 consensus_queue.cc:237] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } }
I20260809 00:42:18.357996 962 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: TxnStatusTablet state changed. Reason: New leader d0d53ad2ccdd4ed8a325479a70f9f351. Latest consensus state: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.358052 962 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:18.358326 1009 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:42:18.358477 1009 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:42:18.358520 32728 catalog_manager.cc:5495] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0d53ad2ccdd4ed8a325479a70f9f351 (127.30.104.193). New cstate: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:18.367712 1002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.365059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:37799 (local address 127.0.0.1:42238)
0809 00:42:18.365134 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:18.365146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:18.365202 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:18.365378 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:18.365378 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:18.365379 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:18.365441 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.365444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.366123 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.366124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.366723 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.366725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.367115 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.367116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.367188 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.367210 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:18.367520 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:18.367531 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 00:42:18.367588 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:42:18.367642 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:42:18.367699 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.365146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42238 (local address 127.30.104.193:37799)
0809 00:42:18.365199 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:18.365202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:18.365211 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:18.365284 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:18.365285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:18.365305 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:18.365364 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:18.365520 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.366033 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.366815 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.367039 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.367116 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.367308 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:18.367424 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:18.367535 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:18.367594 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 00:42:18.367618 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:42:18.369130 670 raft_consensus.cc:1261] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Refusing update from remote peer d0d53ad2ccdd4ed8a325479a70f9f351: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:18.369274 861 raft_consensus.cc:1261] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Refusing update from remote peer d0d53ad2ccdd4ed8a325479a70f9f351: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:18.369529 995 consensus_queue.cc:1035] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:18.369663 962 consensus_queue.cc:1035] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:18.370304 980 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: TxnStatusTablet state changed. Reason: New leader d0d53ad2ccdd4ed8a325479a70f9f351. Latest consensus state: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.370291 966 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: TxnStatusTablet state changed. Reason: New leader d0d53ad2ccdd4ed8a325479a70f9f351. Latest consensus state: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.370402 980 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:18.370402 966 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:18.371135 962 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.371218 962 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:18.371788 995 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.371881 995 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P d0d53ad2ccdd4ed8a325479a70f9f351: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:18.372185 972 mvcc.cc:205] Tried to move back new op lower bound from 7316423222757142528 to 7316423222713315328. Current Snapshot: MvccSnapshot[applied={T|T < 7316423222757142528}]
I20260809 00:42:18.372290 980 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.372377 980 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P b346f870505d4b17a05448ffb66a1929: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:18.372776 966 tablet_replica.cc:439] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0d53ad2ccdd4ed8a325479a70f9f351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37799 } } peers { permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 } } peers { permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 } } }
I20260809 00:42:18.372869 966 tablet_replica.cc:442] T cee871e7eb4942b7a56591a6f10641c0 P d73d61096b444efa9e84107691c4870c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:18.375545 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.373283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42246 (local address 127.30.104.193:37799)
0809 00:42:18.373321 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:18.373323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:18.373332 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:18.373355 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:18.373356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:18.373365 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:18.373417 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:18.373599 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.374068 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.374793 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.375001 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.375076 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.375233 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:18.375267 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:18.375351 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:18.375414 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 00:42:18.375472 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:42:18.381407 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.378300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:37617 (local address 127.30.104.193:37799)
0809 00:42:18.378365 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:18.378367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:18.378489 (+ 122us) server_negotiation.cc:407] Connection header received
0809 00:42:18.378533 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:18.378533 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:18.378544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:18.378612 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:18.378722 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.379459 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.380441 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.380748 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.380823 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.381013 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:18.381049 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:18.381120 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:18.381244 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 00:42:18.381274 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":45}
I20260809 00:42:18.381500 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.378234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:37799 (local address 127.30.104.193:37617)
0809 00:42:18.378411 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:18.378422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:18.378489 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:18.378614 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:18.378615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:18.378616 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:18.378665 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.378666 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.379562 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.379564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.380356 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:18.380358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.380841 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.380842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:18.380893 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.380911 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:18.381168 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:18.381171 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:18.381232 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:42:18.381277 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260809 00:42:18.384109 861 raft_consensus.cc:1261] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Refusing update from remote peer d0d53ad2ccdd4ed8a325479a70f9f351: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:18.384100 670 raft_consensus.cc:1261] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Refusing update from remote peer d0d53ad2ccdd4ed8a325479a70f9f351: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:18.384392 1019 consensus_queue.cc:1035] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b346f870505d4b17a05448ffb66a1929" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:18.384542 962 consensus_queue.cc:1035] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d73d61096b444efa9e84107691c4870c" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:18.404942 32726 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:18.405062 32726 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59202:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:18.405774 32726 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93b10f038d554e95b9272b66bc46b4dc: Sending DeleteTablet for 3 replicas of tablet decfa58805a241a49ceb7b6e45f147d8
I20260809 00:42:18.406040 453 tablet_service.cc:1514] Processing DeleteTablet for tablet decfa58805a241a49ceb7b6e45f147d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:18 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:42224
I20260809 00:42:18.406072 837 tablet_service.cc:1514] Processing DeleteTablet for tablet decfa58805a241a49ceb7b6e45f147d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:18 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:52728
I20260809 00:42:18.406158 986 tablet_replica.cc:330] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: stopping tablet replica
I20260809 00:42:18.406178 643 tablet_service.cc:1514] Processing DeleteTablet for tablet decfa58805a241a49ceb7b6e45f147d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:18 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:60748
I20260809 00:42:18.406239 986 raft_consensus.cc:2229] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:18.406270 985 tablet_replica.cc:330] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: stopping tablet replica
I20260809 00:42:18.406332 985 raft_consensus.cc:2229] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:18.406354 986 raft_consensus.cc:2258] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:18.406421 985 raft_consensus.cc:2258] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:18.406502 987 tablet_replica.cc:330] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: stopping tablet replica
I20260809 00:42:18.406556 987 raft_consensus.cc:2229] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:18.406602 987 raft_consensus.cc:2258] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:18.406783 986 ts_tablet_manager.cc:1905] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:18.406967 987 ts_tablet_manager.cc:1905] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:18.406968 985 ts_tablet_manager.cc:1905] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:18.408674 985 ts_tablet_manager.cc:1918] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:18.408737 985 log.cc:1196] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/decfa58805a241a49ceb7b6e45f147d8
I20260809 00:42:18.408960 985 ts_tablet_manager.cc:1939] T decfa58805a241a49ceb7b6e45f147d8 P b346f870505d4b17a05448ffb66a1929: Deleting consensus metadata
I20260809 00:42:18.408962 986 ts_tablet_manager.cc:1918] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:18.409433 32710 catalog_manager.cc:4841] TS b346f870505d4b17a05448ffb66a1929 (127.30.104.195:35701): tablet decfa58805a241a49ceb7b6e45f147d8 (table test-table [id=81e2eecf19264b57bbce91321f42da37]) successfully deleted
I20260809 00:42:18.409983 986 log.cc:1196] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/decfa58805a241a49ceb7b6e45f147d8
I20260809 00:42:18.410243 986 ts_tablet_manager.cc:1939] T decfa58805a241a49ceb7b6e45f147d8 P d0d53ad2ccdd4ed8a325479a70f9f351: Deleting consensus metadata
I20260809 00:42:18.410566 32709 catalog_manager.cc:4841] TS d0d53ad2ccdd4ed8a325479a70f9f351 (127.30.104.193:37799): tablet decfa58805a241a49ceb7b6e45f147d8 (table test-table [id=81e2eecf19264b57bbce91321f42da37]) successfully deleted
I20260809 00:42:18.411244 987 ts_tablet_manager.cc:1918] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:18.411334 987 log.cc:1196] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/decfa58805a241a49ceb7b6e45f147d8
I20260809 00:42:18.411580 987 ts_tablet_manager.cc:1939] T decfa58805a241a49ceb7b6e45f147d8 P d73d61096b444efa9e84107691c4870c: Deleting consensus metadata
I20260809 00:42:18.411934 32709 catalog_manager.cc:4841] TS d73d61096b444efa9e84107691c4870c (127.30.104.194:46065): tablet decfa58805a241a49ceb7b6e45f147d8 (table test-table [id=81e2eecf19264b57bbce91321f42da37]) successfully deleted
Aug 09 00:42:18 dist-test-slave-c12x krb5kdc[32668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:18.420603 950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.410545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.30.104.193:37799)
0809 00:42:18.410590 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:18.410592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:18.410598 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:42:18.410619 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:18.410620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:18.410633 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:18.410721 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:18.411467 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.412220 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.413321 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.413527 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.413600 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.417407 (+ 3807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:18.417420 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:18.417421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:18.417430 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:18.419240 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:18.419712 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:18.419716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:18.419716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:18.419784 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:18.419995 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:18.419996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:18.419997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:18.420157 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:18.420338 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:18.420378 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 00:42:18.420468 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 00:42:18.420913 541 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42258
I20260809 00:42:18.428737 337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:18.421931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59216 (local address 127.30.104.254:37853)
0809 00:42:18.421987 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:18.421989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:18.421999 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:18.422046 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:18.422047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:18.422058 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:18.422102 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:18.422626 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.423140 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.423933 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:18.424152 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:18.424238 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:18.425479 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:18.425481 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:18.425481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:18.425488 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:18.427320 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:18.427819 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:18.427821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:18.427821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:18.427894 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:18.428127 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:18.428128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:18.428128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:18.428274 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:18.428385 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:18.428530 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:42:18.428622 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 00:42:18.429008 32726 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59216
I20260809 00:42:18.429875 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 367
I20260809 00:42:18.455791 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 564
I20260809 00:42:18.461853 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 748
I20260809 00:42:18.473871 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 32685
2026-08-09T00:42:18Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2815 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:18 dist-test-slave-c12x krb5kdc[1039](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:18 dist-test-slave-c12x krb5kdc[1039](info): set up 2 sockets
Aug 09 00:42:18 dist-test-slave-c12x krb5kdc[1039](info): commencing operation
krb5kdc: starting...
W20260809 00:42:19.723898 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:19 dist-test-slave-c12x krb5kdc[1039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:19.859130 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33225
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:33225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:19.948277 1053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:19.948565 1053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:19.948609 1053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:19.949846 1053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:19.949930 1053 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:42:19.949954 1053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:19.949975 1053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:19.950032 1053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:19.950063 1053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:19.951730 1053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:33225
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:19.952152 1053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:19.952486 1053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:19.958128 1060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:19.958175 1063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:19.958174 1059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:19.958576 1053 server_base.cc:1034] running on GCE node
I20260809 00:42:19.958822 1053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:19.959100 1053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:19.960292 1053 hybrid_clock.cc:648] HybridClock initialized: now 1786236139960296 us; error 50 us; skew 500 ppm
Aug 09 00:42:19 dist-test-slave-c12x krb5kdc[1039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236139, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:19.963016 1053 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:19.963619 1053 webserver.cc:460] Webserver started at http://127.30.104.254:44551/ using document root <none> and password file <none>
I20260809 00:42:19.963891 1053 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:19.963966 1053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:19.964134 1053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:19.965312 1053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689"
format_stamp: "Formatted at 2026-08-09 00:42:19 on dist-test-slave-c12x"
server_key: "189661f8bbaf425fe5151f393a800657"
server_key_iv: "ecbd4fff2a357e3d8ffd2b06a88c8c89"
server_key_version: "encryptionkey@0"
I20260809 00:42:19.965723 1053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689"
format_stamp: "Formatted at 2026-08-09 00:42:19 on dist-test-slave-c12x"
server_key: "189661f8bbaf425fe5151f393a800657"
server_key_iv: "ecbd4fff2a357e3d8ffd2b06a88c8c89"
server_key_version: "encryptionkey@0"
I20260809 00:42:19.967235 1053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:19.968277 1070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:19.968562 1053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:19.968727 1053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689"
format_stamp: "Formatted at 2026-08-09 00:42:19 on dist-test-slave-c12x"
server_key: "189661f8bbaf425fe5151f393a800657"
server_key_iv: "ecbd4fff2a357e3d8ffd2b06a88c8c89"
server_key_version: "encryptionkey@0"
I20260809 00:42:19.968817 1053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:19.981135 1053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:19.983805 1053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:19.983964 1053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:19.989012 1053 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:33225
I20260809 00:42:19.989179 1157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:33225 every 8 connection(s)
I20260809 00:42:19.989498 1053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:19.990304 1158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:19.992849 1158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: Bootstrap starting.
I20260809 00:42:19.993469 1158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:19.993726 1158 log.cc:824] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:19.994526 1158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: No bootstrap required, opened a new log
I20260809 00:42:19.995556 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 1053
I20260809 00:42:19.995666 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:19.995849 31139 external_mini_cluster.cc:1428] Setting key 32bc4bd291856875cf3f351310aa2c7d
I20260809 00:42:19.996768 1158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33225 } }
I20260809 00:42:19.996985 1158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:19.997031 1158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5157724a8b6d4a6fbecfbcd3b5b3e689, State: Initialized, Role: FOLLOWER
I20260809 00:42:19.997212 1158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33225 } }
I20260809 00:42:19.997349 1158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:19.997404 1158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:19.997459 1158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:19.998440 1158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33225 } }
I20260809 00:42:19.998592 1158 leader_election.cc:302] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5157724a8b6d4a6fbecfbcd3b5b3e689; no voters:
I20260809 00:42:19.998797 1158 leader_election.cc:288] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:19.998826 1164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:19.998943 1164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [term 1 LEADER]: Becoming Leader. State: Replica: 5157724a8b6d4a6fbecfbcd3b5b3e689, State: Running, Role: LEADER
I20260809 00:42:19.999056 1164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33225 } }
I20260809 00:42:19.999091 1158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:19.999728 1165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5157724a8b6d4a6fbecfbcd3b5b3e689. Latest consensus state: current_term: 1 leader_uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33225 } } }
I20260809 00:42:19.999830 1165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:19.999909 1167 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5157724a8b6d4a6fbecfbcd3b5b3e689" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33225 } } }
I20260809 00:42:20.000160 1167 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:20.000149 1177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:20.000798 1177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:20.002615 1177 catalog_manager.cc:1347] Generated new cluster ID: 8f2d01086f1d4628bcbd698b3df0899c
I20260809 00:42:20.002722 1177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:20.008175 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:19.996815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.30.104.254:33225)
0809 00:42:19.997112 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:19.997139 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:19.997177 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:42:19.997307 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:19.997313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:19.997385 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:19.997579 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:19.998539 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.000169 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.001351 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.001948 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.002097 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.004616 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.004621 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.004621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.004633 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.006366 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.006880 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.006883 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.006883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.007002 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.007242 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.007243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.007243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.007577 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.007766 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.007840 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.007990 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":134,"threads_started":1}
I20260809 00:42:20.040185 1177 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:20.040953 1177 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:20.044642 1177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: Generated new TSK 0
I20260809 00:42:20.044863 1177 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:20.081979 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33225
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:20.200416 1194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:20.200646 1194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:20.200675 1194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:20.201881 1194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:20.201926 1194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:20.201947 1194 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:42:20.201960 1194 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:20.202006 1194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:20.203596 1194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:33225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:20.203855 1194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:20.204123 1194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:20.210232 1204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:20.210215 1199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:20.210333 1194 server_base.cc:1034] running on GCE node
W20260809 00:42:20.210215 1202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:20.210696 1194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:20.210978 1194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:20.212136 1194 hybrid_clock.cc:648] HybridClock initialized: now 1786236140212122 us; error 32 us; skew 500 ppm
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236140, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:20.214839 1194 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:20.215255 1194 webserver.cc:460] Webserver started at http://127.30.104.193:44741/ using document root <none> and password file <none>
I20260809 00:42:20.215509 1194 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:20.215566 1194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:20.215664 1194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:20.216729 1194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "ce3c656803684b3c85151645dc488adf"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "3698ac7609d3cd7ecd67679a57c815a3"
server_key_iv: "932d62cbbba8f259ded7622f1c8cc12a"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.217054 1194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "ce3c656803684b3c85151645dc488adf"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "3698ac7609d3cd7ecd67679a57c815a3"
server_key_iv: "932d62cbbba8f259ded7622f1c8cc12a"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.218474 1194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:20.219308 1213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:20.219553 1194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:20.219681 1194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "ce3c656803684b3c85151645dc488adf"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "3698ac7609d3cd7ecd67679a57c815a3"
server_key_iv: "932d62cbbba8f259ded7622f1c8cc12a"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.219767 1194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:20.236001 1194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:20.238421 1194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:20.238585 1194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:20.239241 1194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:20.239283 1194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:20.239305 1194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:20.239318 1194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:20.246443 1194 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:44535
I20260809 00:42:20.246496 1381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:44535 every 8 connection(s)
I20260809 00:42:20.246812 1194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236140, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:20.249353 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 1194
I20260809 00:42:20.249444 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:20.249640 31139 external_mini_cluster.cc:1428] Setting key 1cb2865c23f9e754e74d4db07de23f89
I20260809 00:42:20.250834 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.239817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:53185 (local address 127.30.104.254:33225)
0809 00:42:20.239875 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.239877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.240110 (+ 233us) server_negotiation.cc:407] Connection header received
0809 00:42:20.240646 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.240647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.240665 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.240747 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:20.241435 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.242303 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.244129 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.244521 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.244597 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.247547 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.247553 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.247553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.247561 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.249042 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.249538 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.249539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.249540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.249626 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.249984 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.249986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.249986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.250135 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.250213 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.250620 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.250735 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:42:20.251029 1225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.239799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33225 (local address 127.30.104.193:53185)
0809 00:42:20.239990 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.240027 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.240537 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.240800 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.240803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.241119 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:20.241312 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.241317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.242435 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.242437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.244011 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.244015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.244632 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.244633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.244767 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.245407 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:20.245418 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:20.247381 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:20.249143 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.249151 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.249153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.249441 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.249708 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.249709 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.249710 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.249855 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.250242 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:20.250251 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:20.250454 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.250576 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.250758 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":109,"threads_started":1}
I20260809 00:42:20.251621 1386 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33225
I20260809 00:42:20.251744 1386 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:20.251971 1386 heartbeater.cc:511] Master 127.30.104.254:33225 requested a full tablet report, sending...
I20260809 00:42:20.252532 1095 ts_manager.cc:194] Registered new tserver with Master: ce3c656803684b3c85151645dc488adf (127.30.104.193:44535)
I20260809 00:42:20.253247 1095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:53185
I20260809 00:42:20.259963 1396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.252790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33225 (local address 127.30.104.193:60685)
0809 00:42:20.252947 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.252958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.253019 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.253163 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.253163 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.253418 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:20.253488 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.253511 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.254551 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.254552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.255130 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.255132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.255664 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.255665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.255747 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.256274 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:20.256279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:20.256625 (+ 346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:20.258669 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.258670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.258670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.258954 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.259180 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.259181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.259181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.259240 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.259630 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:20.259631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:20.259687 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.259763 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.259884 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260809 00:42:20.260175 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.252852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:60685 (local address 127.30.104.254:33225)
0809 00:42:20.252923 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.252926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.253002 (+ 76us) server_negotiation.cc:407] Connection header received
0809 00:42:20.253085 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.253086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.253096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.253157 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:20.253588 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.254432 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.255237 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.255577 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.255654 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.256728 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.256730 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.256731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.256739 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.258562 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.259023 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.259024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.259025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.259101 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.259312 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.259313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.259314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.259530 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.259613 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.259814 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.259917 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:20.281694 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33225
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:20.368777 1398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:20.369055 1398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:20.369127 1398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:20.370404 1398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:20.370479 1398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:20.370520 1398 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:42:20.370568 1398 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:20.370609 1398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:20.372386 1398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:33225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:20.372706 1398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:20.372965 1398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:20.378477 1406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:20.378449 1404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:20.378490 1398 server_base.cc:1034] running on GCE node
W20260809 00:42:20.378448 1403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:20.379034 1398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:20.379320 1398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:20.380505 1398 hybrid_clock.cc:648] HybridClock initialized: now 1786236140380485 us; error 35 us; skew 500 ppm
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236140, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:20.383121 1398 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:20.383685 1398 webserver.cc:460] Webserver started at http://127.30.104.194:46361/ using document root <none> and password file <none>
I20260809 00:42:20.383948 1398 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:20.384022 1398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:20.384174 1398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:20.385278 1398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "e19e55899e2c4455aaab62be315b0564"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "443c17b48b20d27d86debf6ae3c00372"
server_key_iv: "d4340657a10a54f9641997e772a062cc"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.385664 1398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "e19e55899e2c4455aaab62be315b0564"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "443c17b48b20d27d86debf6ae3c00372"
server_key_iv: "d4340657a10a54f9641997e772a062cc"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.387355 1398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:20.388345 1416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:20.388573 1398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:20.388687 1398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "e19e55899e2c4455aaab62be315b0564"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "443c17b48b20d27d86debf6ae3c00372"
server_key_iv: "d4340657a10a54f9641997e772a062cc"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.388774 1398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:20.400095 1398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:20.402446 1398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:20.402617 1398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:20.403168 1398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:20.403219 1398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:20.403244 1398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:20.403296 1398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:20.410717 1398 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:36745
I20260809 00:42:20.410794 1588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:36745 every 8 connection(s)
I20260809 00:42:20.411113 1398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236140, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:20.414808 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.403810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:52417 (local address 127.30.104.254:33225)
0809 00:42:20.403892 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.403895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.404091 (+ 196us) server_negotiation.cc:407] Connection header received
0809 00:42:20.404609 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.404611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.404626 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.404711 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:20.405774 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.406630 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.408521 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.408812 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.408888 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.411836 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.411838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.411838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.411846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.413330 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.413775 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.413777 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.413777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.413846 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.414126 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.414127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.414127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.414255 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.414331 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.414624 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.414712 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:20.415015 1427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.403764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33225 (local address 127.30.104.194:52417)
0809 00:42:20.403951 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.403992 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.404502 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.404873 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.404878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.405316 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:20.405625 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.405631 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.406787 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.406788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.408382 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.408389 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.408926 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.408927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.409131 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.409817 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:20.409830 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:20.411728 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:20.413449 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.413455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.413457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.413689 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.413918 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.413919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.413919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.414048 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.414322 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:20.414329 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:20.414484 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.414599 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.414774 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260809 00:42:20.415607 1589 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33225
I20260809 00:42:20.415722 1589 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:20.415913 1589 heartbeater.cc:511] Master 127.30.104.254:33225 requested a full tablet report, sending...
I20260809 00:42:20.416363 1095 ts_manager.cc:194] Registered new tserver with Master: e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745)
I20260809 00:42:20.416929 1095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:52417
I20260809 00:42:20.417883 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 1398
I20260809 00:42:20.417971 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:20.418160 31139 external_mini_cluster.cc:1428] Setting key 6e163d9ea10af857acf49540c9ea2958
I20260809 00:42:20.422879 1595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.416734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33225 (local address 127.30.104.194:52003)
0809 00:42:20.416867 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.416884 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.416945 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.417081 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.417083 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.417313 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:20.417385 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.417387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.418103 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.418103 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.418651 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.418653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.419036 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.419037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.419111 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.419588 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:20.419593 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:20.419885 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:20.421696 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.421697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.421697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.421869 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.422146 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.422147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.422147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.422223 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.422596 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:20.422596 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:20.422637 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.422705 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.422793 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260809 00:42:20.422968 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.416806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:52003 (local address 127.30.104.254:33225)
0809 00:42:20.416855 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.416857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.416928 (+ 71us) server_negotiation.cc:407] Connection header received
0809 00:42:20.417005 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.417006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.417016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.417068 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:20.417472 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.417983 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.418739 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.418949 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.419044 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.420003 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.420004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.420005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.420011 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.421607 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.421947 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.421948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.421949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.422010 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.422298 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.422299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.422299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.422517 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.422589 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.422751 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.422840 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:20.447041 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33225
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:46531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:20.536675 1598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:20.536942 1598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:20.536986 1598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:20.538193 1598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:20.538275 1598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:20.538311 1598 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 00:42:20.538343 1598 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:20.538384 1598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:20.540043 1598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:46531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:33225
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:20.540359 1598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:20.540627 1598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:20.545979 1607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:20.546008 1604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:20.546048 1605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:20.546216 1598 server_base.cc:1034] running on GCE node
I20260809 00:42:20.546574 1598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:20.546830 1598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:20.548008 1598 hybrid_clock.cc:648] HybridClock initialized: now 1786236140547993 us; error 38 us; skew 500 ppm
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236140, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:20.550889 1598 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:20.551504 1598 webserver.cc:460] Webserver started at http://127.30.104.195:41837/ using document root <none> and password file <none>
I20260809 00:42:20.551754 1598 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:20.551827 1598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:20.552045 1598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:20.553207 1598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "e1efc904c7764a46bb08655f1d54971d"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "52de5ebf1dcf4d251ed8e2dab03a671f"
server_key_iv: "f1cd43500549a7cf503d7899b9976445"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.553607 1598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "e1efc904c7764a46bb08655f1d54971d"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "52de5ebf1dcf4d251ed8e2dab03a671f"
server_key_iv: "f1cd43500549a7cf503d7899b9976445"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.555092 1598 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:20.556182 1618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:20.556442 1598 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:20.556571 1598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "e1efc904c7764a46bb08655f1d54971d"
format_stamp: "Formatted at 2026-08-09 00:42:20 on dist-test-slave-c12x"
server_key: "52de5ebf1dcf4d251ed8e2dab03a671f"
server_key_iv: "f1cd43500549a7cf503d7899b9976445"
server_key_version: "encryptionkey@0"
I20260809 00:42:20.556651 1598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:20.584198 1598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:20.587385 1598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:20.587591 1598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:20.588327 1598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:20.588404 1598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:20.588446 1598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:20.588477 1598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236140, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:20.596936 1598 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:34399
I20260809 00:42:20.597425 1598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:20.597775 1779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:34399 every 8 connection(s)
I20260809 00:42:20.599771 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.589200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:45411 (local address 127.30.104.254:33225)
0809 00:42:20.589248 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.589250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.589260 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:20.589759 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.589760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.589773 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.589833 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:20.590724 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.591571 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.593143 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.593499 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.593571 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.596307 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.596308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.596309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.596317 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.598100 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.598624 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.598626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.598626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.598692 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.598972 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.598973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.598973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.599105 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.599183 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.599579 (+ 396us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.599679 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:42:20.599973 1627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.588820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33225 (local address 127.30.104.195:45411)
0809 00:42:20.589014 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.589085 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.589638 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.589910 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.589914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.590250 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:20.590591 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.590598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.591695 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.591696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.593062 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.593065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.593598 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.593599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.593746 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.594360 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:20.594370 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:20.596215 (+ 1845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:20.598199 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.598206 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.598208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.598557 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.598773 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.598774 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.598774 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.598892 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.599174 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:20.599182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:20.599378 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.599515 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.599718 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
I20260809 00:42:20.600943 1780 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33225
I20260809 00:42:20.601044 1780 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:20.601205 1780 heartbeater.cc:511] Master 127.30.104.254:33225 requested a full tablet report, sending...
I20260809 00:42:20.601595 1096 ts_manager.cc:194] Registered new tserver with Master: e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:20.602046 1096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:45411
I20260809 00:42:20.603664 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 1598
I20260809 00:42:20.603750 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:20.603889 31139 external_mini_cluster.cc:1428] Setting key 78f4749537e5670f34f2c8f09a104d35
I20260809 00:42:20.604710 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:42:20.610652 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.602321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:45567 (local address 127.30.104.254:33225)
0809 00:42:20.602362 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.602364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.603070 (+ 706us) server_negotiation.cc:407] Connection header received
0809 00:42:20.603148 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.603149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.603159 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.603210 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:20.603650 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.604519 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.605893 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.606248 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.606293 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.607715 (+ 1422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.607716 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.607717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.607726 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.609384 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.609835 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.609836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.609836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.609893 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.610111 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.610112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.610112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.610221 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.610274 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.610514 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.610583 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:42:20.610913 1789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.602824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33225 (local address 127.30.104.195:45567)
0809 00:42:20.602983 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.602995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.603085 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.603213 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.603213 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.603469 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:20.603556 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.603559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.604720 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.604721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.605772 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.605775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.606389 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.606390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.606470 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.607054 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:20.607060 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:20.607597 (+ 537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:20.609479 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.609481 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.609481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.609747 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.609965 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.609966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.609967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.610045 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.610288 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:20.610289 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:20.610357 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.610451 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.610594 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":99,"threads_started":1}
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:20.637377 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.628001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.30.104.254:33225)
0809 00:42:20.628048 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.628051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.628061 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:20.628153 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.628154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.628164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.628221 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:20.629108 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.629798 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.631105 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.631300 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.631364 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.634485 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.634486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.634487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.634495 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.636067 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.636530 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.636531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.636531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.636590 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.636843 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.636845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.636845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.636986 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.637079 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.637236 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.637306 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:20.639694 1096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:20.642616 1096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:20.650915 1804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.645430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:36745 (local address 127.0.0.1:38330)
0809 00:42:20.645577 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.645621 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.645690 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.646583 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.646588 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.646590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:20.646678 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.646681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.648381 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.648381 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.649277 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.649280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.650596 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.650597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.650701 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.650719 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.650769 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.650821 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":28,"thread_start_us":71,"threads_started":1}
I20260809 00:42:20.651459 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.645811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.30.104.194:36745)
0809 00:42:20.646387 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.646393 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.646404 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:20.646447 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.646449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.646466 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.646559 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:20.646772 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.648296 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.649378 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.650170 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.651194 (+ 1024us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.651259 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.651285 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.651327 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":143,"threads_started":1}
I20260809 00:42:20.651579 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.645659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:34399 (local address 127.0.0.1:49836)
0809 00:42:20.645865 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.645873 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.645911 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.646780 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.646781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.646783 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:20.646828 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.646829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.648352 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.648353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.649804 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.649807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.651175 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.651177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.651260 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.651266 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.651324 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.651377 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":30,"thread_start_us":84,"threads_started":1}
I20260809 00:42:20.651945 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.646221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.30.104.195:34399)
0809 00:42:20.646402 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.646405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.646416 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:20.646450 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.646452 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.646466 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.646563 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:20.646910 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.648229 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.649922 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.650825 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.651038 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.651126 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.651804 (+ 678us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.651853 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":107,"threads_started":1}
I20260809 00:42:20.652465 1805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.645468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:44535 (local address 127.0.0.1:43918)
0809 00:42:20.645719 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.645730 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.645784 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.646890 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.646891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.646894 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:20.646946 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.646948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.648279 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.648280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.650387 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.650389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.652137 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.652138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.652269 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.652274 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.652356 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.652396 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":46,"thread_start_us":104,"threads_started":1}
I20260809 00:42:20.652334 1488 tablet_service.cc:1467] Processing CreateTablet for tablet 30af236e869c4a7cb9f455da1b53647c (DEFAULT_TABLE table=test-table [id=5034d4f682be48b79f26dfa45f731f6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:20.652498 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.646150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.30.104.193:44535)
0809 00:42:20.646693 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.646697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.646704 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:42:20.646757 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.646760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.646775 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.647019 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:20.647093 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.648188 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.650480 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.651927 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.652004 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.652077 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.652377 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.652418 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":81,"threads_started":1}
I20260809 00:42:20.652534 1689 tablet_service.cc:1467] Processing CreateTablet for tablet 30af236e869c4a7cb9f455da1b53647c (DEFAULT_TABLE table=test-table [id=5034d4f682be48b79f26dfa45f731f6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:20.652740 1488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30af236e869c4a7cb9f455da1b53647c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:20.652832 1689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30af236e869c4a7cb9f455da1b53647c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:20.653391 1284 tablet_service.cc:1467] Processing CreateTablet for tablet 30af236e869c4a7cb9f455da1b53647c (DEFAULT_TABLE table=test-table [id=5034d4f682be48b79f26dfa45f731f6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:20.653733 1284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30af236e869c4a7cb9f455da1b53647c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:20.656327 1812 tablet_bootstrap.cc:492] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Bootstrap starting.
I20260809 00:42:20.656692 1814 tablet_bootstrap.cc:492] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Bootstrap starting.
I20260809 00:42:20.657449 1814 tablet_bootstrap.cc:654] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:20.657677 1812 tablet_bootstrap.cc:654] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:20.657823 1814 log.cc:824] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:20.658048 1812 log.cc:824] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:20.658696 1814 tablet_bootstrap.cc:492] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: No bootstrap required, opened a new log
I20260809 00:42:20.658782 1814 ts_tablet_manager.cc:1397] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:20.660101 1812 tablet_bootstrap.cc:492] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: No bootstrap required, opened a new log
I20260809 00:42:20.660187 1812 ts_tablet_manager.cc:1397] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 00:42:20.661484 1814 raft_consensus.cc:348] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.661691 1814 raft_consensus.cc:374] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:20.661741 1814 raft_consensus.cc:729] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19e55899e2c4455aaab62be315b0564, State: Initialized, Role: FOLLOWER
I20260809 00:42:20.661883 1814 consensus_queue.cc:260] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.661917 1812 raft_consensus.cc:348] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.662145 1812 raft_consensus.cc:374] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:20.662197 1812 raft_consensus.cc:729] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1efc904c7764a46bb08655f1d54971d, State: Initialized, Role: FOLLOWER
I20260809 00:42:20.662197 1814 ts_tablet_manager.cc:1428] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:42:20.662339 1589 heartbeater.cc:503] Master 127.30.104.254:33225 was elected leader, sending a full tablet report...
I20260809 00:42:20.662321 1812 consensus_queue.cc:260] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.662590 1812 ts_tablet_manager.cc:1428] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:42:20.662619 1780 heartbeater.cc:503] Master 127.30.104.254:33225 was elected leader, sending a full tablet report...
I20260809 00:42:20.667099 1815 tablet_bootstrap.cc:492] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Bootstrap starting.
I20260809 00:42:20.667768 1815 tablet_bootstrap.cc:654] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:20.668047 1815 log.cc:824] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:20.668846 1815 tablet_bootstrap.cc:492] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: No bootstrap required, opened a new log
I20260809 00:42:20.668955 1815 ts_tablet_manager.cc:1397] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:20.670544 1815 raft_consensus.cc:348] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.670713 1815 raft_consensus.cc:374] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:20.670740 1815 raft_consensus.cc:729] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3c656803684b3c85151645dc488adf, State: Initialized, Role: FOLLOWER
I20260809 00:42:20.670883 1815 consensus_queue.cc:260] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.671173 1815 ts_tablet_manager.cc:1428] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 00:42:20.671216 1386 heartbeater.cc:503] Master 127.30.104.254:33225 was elected leader, sending a full tablet report...
W20260809 00:42:20.747701 1387 tablet.cc:2369] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:42:20.848641 1781 tablet.cc:2369] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:42:20.891142 1825 raft_consensus.cc:482] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:20.891371 1825 raft_consensus.cc:504] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.891846 1825 leader_election.cc:288] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745), e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:20.896788 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.892580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:34005 (local address 127.30.104.195:34399)
0809 00:42:20.892626 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.892634 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.892641 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:42:20.892696 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.892697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.892703 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.892783 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:20.892925 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.893628 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.895776 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.896353 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.896444 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.896453 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.896666 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.896713 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:42:20.896788 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.892218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:53605 (local address 127.30.104.194:36745)
0809 00:42:20.892290 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.892293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.892389 (+ 96us) server_negotiation.cc:407] Connection header received
0809 00:42:20.892456 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.892458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.892470 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.892549 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:20.892775 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.893615 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.895776 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.896369 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.896444 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.896453 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.896666 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.896713 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:42:20.896823 1827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.892127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:36745 (local address 127.30.104.193:53605)
0809 00:42:20.892271 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.892285 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.892379 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.892614 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.892615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.892618 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:20.892688 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.892690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.893910 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.893912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.895646 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.895650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.896463 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.896464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.896552 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.896557 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.896626 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.896685 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":186,"thread_start_us":98,"threads_started":1}
I20260809 00:42:20.896823 1828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.892434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:34399 (local address 127.30.104.193:34005)
0809 00:42:20.892574 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.892585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.892632 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.892785 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.892786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.892788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:20.892844 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.892845 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.893929 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.893930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.895646 (+ 1716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.895650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.896462 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.896464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.896580 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.896583 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.896638 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.896685 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":51,"thread_start_us":71,"threads_started":1}
I20260809 00:42:20.897295 1712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30af236e869c4a7cb9f455da1b53647c" candidate_uuid: "ce3c656803684b3c85151645dc488adf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1efc904c7764a46bb08655f1d54971d" is_pre_election: true
I20260809 00:42:20.897490 1712 raft_consensus.cc:2454] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce3c656803684b3c85151645dc488adf in term 0.
I20260809 00:42:20.897533 1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30af236e869c4a7cb9f455da1b53647c" candidate_uuid: "ce3c656803684b3c85151645dc488adf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19e55899e2c4455aaab62be315b0564" is_pre_election: true
I20260809 00:42:20.897711 1518 raft_consensus.cc:2454] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce3c656803684b3c85151645dc488adf in term 0.
I20260809 00:42:20.897778 1215 leader_election.cc:302] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3c656803684b3c85151645dc488adf, e1efc904c7764a46bb08655f1d54971d; no voters:
I20260809 00:42:20.897917 1825 raft_consensus.cc:2781] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:20.898006 1825 raft_consensus.cc:482] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:20.898048 1825 raft_consensus.cc:3037] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:20.898887 1825 raft_consensus.cc:504] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.899049 1825 leader_election.cc:288] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [CANDIDATE]: Term 1 election: Requested vote from peers e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745), e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:20.899225 1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30af236e869c4a7cb9f455da1b53647c" candidate_uuid: "ce3c656803684b3c85151645dc488adf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19e55899e2c4455aaab62be315b0564"
I20260809 00:42:20.899267 1712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30af236e869c4a7cb9f455da1b53647c" candidate_uuid: "ce3c656803684b3c85151645dc488adf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1efc904c7764a46bb08655f1d54971d"
I20260809 00:42:20.899334 1518 raft_consensus.cc:3037] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:20.899375 1712 raft_consensus.cc:3037] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:20.900148 1518 raft_consensus.cc:2454] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce3c656803684b3c85151645dc488adf in term 1.
I20260809 00:42:20.900328 1712 raft_consensus.cc:2454] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce3c656803684b3c85151645dc488adf in term 1.
I20260809 00:42:20.900338 1215 leader_election.cc:302] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3c656803684b3c85151645dc488adf, e19e55899e2c4455aaab62be315b0564; no voters:
I20260809 00:42:20.900435 1825 raft_consensus.cc:2781] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:20.900568 1825 raft_consensus.cc:686] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 1 LEADER]: Becoming Leader. State: Replica: ce3c656803684b3c85151645dc488adf, State: Running, Role: LEADER
I20260809 00:42:20.900697 1825 consensus_queue.cc:237] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } }
I20260809 00:42:20.901561 1096 catalog_manager.cc:5495] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf reported cstate change: term changed from 0 to 1, leader changed from <none> to ce3c656803684b3c85151645dc488adf (127.30.104.193). New cstate: current_term: 1 leader_uuid: "ce3c656803684b3c85151645dc488adf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } health_report { overall_health: HEALTHY } } }
W20260809 00:42:20.911911 1590 tablet.cc:2369] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 00:42:20 dist-test-slave-c12x krb5kdc[1039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236139, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:20.970046 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.960323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33225 (local address 127.0.0.1:38412)
0809 00:42:20.960556 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:20.960569 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:20.960653 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:20.960777 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:20.960777 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:20.961199 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:20.961248 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.961250 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.962137 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.962138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.962943 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:20.962947 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.963425 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.963426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:20.963515 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.964209 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:20.964217 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:20.966230 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:20.968489 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.968500 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.968502 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.968862 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.969124 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:20.969124 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:20.969124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:20.969208 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:20.969543 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:20.969552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:20.969631 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 00:42:20.969723 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 00:42:20.969876 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":157,"threads_started":1}
I20260809 00:42:20.972205 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:20.960400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38412 (local address 127.30.104.254:33225)
0809 00:42:20.960476 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:20.960479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:20.960629 (+ 150us) server_negotiation.cc:407] Connection header received
0809 00:42:20.960717 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:20.960717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:20.960727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:20.960773 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:20.961336 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.962041 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.963057 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:20.963328 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:20.963420 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:20.966379 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:20.966381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:20.966381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:20.966391 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:20.968355 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.968957 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.968959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.968959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.969031 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:20.969305 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:20.969306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:20.969306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:20.969467 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:20.969557 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:20.969767 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 00:42:20.972078 (+ 2311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2200,"server-negotiator.queue_time_us":41}
I20260809 00:42:20.975183 1096 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38412:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:42:20.975466 1096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:20.977170 1689 tablet_service.cc:1467] Processing CreateTablet for tablet c6658d95bdd44ebb9126133d8615b279 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8b09c4ac2cd04a7282aa3b9661c465d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:20.977240 1284 tablet_service.cc:1467] Processing CreateTablet for tablet c6658d95bdd44ebb9126133d8615b279 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8b09c4ac2cd04a7282aa3b9661c465d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:20.977295 1488 tablet_service.cc:1467] Processing CreateTablet for tablet c6658d95bdd44ebb9126133d8615b279 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8b09c4ac2cd04a7282aa3b9661c465d7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:20.977378 1284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6658d95bdd44ebb9126133d8615b279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:20.977380 1689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6658d95bdd44ebb9126133d8615b279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:20.977481 1488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6658d95bdd44ebb9126133d8615b279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:20.978950 1815 tablet_bootstrap.cc:492] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: Bootstrap starting.
I20260809 00:42:20.979254 1814 tablet_bootstrap.cc:492] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: Bootstrap starting.
I20260809 00:42:20.979561 1812 tablet_bootstrap.cc:492] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: Bootstrap starting.
I20260809 00:42:20.979692 1814 tablet_bootstrap.cc:654] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:20.979718 1815 tablet_bootstrap.cc:654] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:20.980168 1812 tablet_bootstrap.cc:654] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:20.980432 1814 tablet_bootstrap.cc:492] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: No bootstrap required, opened a new log
I20260809 00:42:20.980479 1814 ts_tablet_manager.cc:1397] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:20.980641 1814 raft_consensus.cc:348] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:20.980727 1814 raft_consensus.cc:374] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:20.980756 1814 raft_consensus.cc:729] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19e55899e2c4455aaab62be315b0564, State: Initialized, Role: FOLLOWER
I20260809 00:42:20.980811 1814 consensus_queue.cc:260] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:20.980878 1812 tablet_bootstrap.cc:492] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: No bootstrap required, opened a new log
I20260809 00:42:20.980931 1812 ts_tablet_manager.cc:1397] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:20.980940 1814 ts_tablet_manager.cc:1428] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:20.981093 1819 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:20.981094 1812 raft_consensus.cc:348] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:20.981170 1812 raft_consensus.cc:374] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:20.981174 1819 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:20.981196 1812 raft_consensus.cc:729] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1efc904c7764a46bb08655f1d54971d, State: Initialized, Role: FOLLOWER
I20260809 00:42:20.981246 1812 consensus_queue.cc:260] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:20.981344 1812 ts_tablet_manager.cc:1428] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:20.981483 1821 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:20.981577 1821 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:20.981639 1815 tablet_bootstrap.cc:492] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: No bootstrap required, opened a new log
I20260809 00:42:20.981690 1815 ts_tablet_manager.cc:1397] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:42:20.981856 1815 raft_consensus.cc:348] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:20.981930 1815 raft_consensus.cc:374] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:20.981948 1815 raft_consensus.cc:729] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3c656803684b3c85151645dc488adf, State: Initialized, Role: FOLLOWER
I20260809 00:42:20.981999 1815 consensus_queue.cc:260] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:20.982103 1815 ts_tablet_manager.cc:1428] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:20.982164 1825 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:20.982241 1825 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:21.132911 1821 raft_consensus.cc:482] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:21.133020 1821 raft_consensus.cc:504] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.133502 1821 leader_election.cc:288] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce3c656803684b3c85151645dc488adf (127.30.104.193:44535), e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745)
I20260809 00:42:21.136569 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.133926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:58629 (local address 127.30.104.193:44535)
0809 00:42:21.133977 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.133979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.133987 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:21.134022 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.134023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.134030 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.134097 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:21.134277 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.134766 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.135822 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.136190 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.136277 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.136286 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.136450 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.136487 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:42:21.136592 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.133944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:58621 (local address 127.30.104.194:36745)
0809 00:42:21.133994 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.133996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.134063 (+ 67us) server_negotiation.cc:407] Connection header received
0809 00:42:21.134115 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.134116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.134126 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.134208 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:21.134369 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.134835 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.135842 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.136198 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.136286 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.136295 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.136483 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.136518 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:42:21.136619 1846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.133715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:44535 (local address 127.30.104.195:58629)
0809 00:42:21.133837 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.133849 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.133914 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.134104 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.134105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.134108 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:21.134195 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.134197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.134856 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.134857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.135724 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.135727 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.136276 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.136277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.136359 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.136363 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.136440 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.136485 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":76,"threads_started":1}
I20260809 00:42:21.136628 1847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.133819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:36745 (local address 127.30.104.195:58621)
0809 00:42:21.133963 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.133975 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.134056 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.134214 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.134215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.134217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:21.134290 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.134292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.134926 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.134927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.135742 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.135745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.136286 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.136287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.136380 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.136383 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.136467 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.136512 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":14,"thread_start_us":68,"threads_started":1}
I20260809 00:42:21.136986 1311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6658d95bdd44ebb9126133d8615b279" candidate_uuid: "e1efc904c7764a46bb08655f1d54971d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce3c656803684b3c85151645dc488adf" is_pre_election: true
I20260809 00:42:21.137144 1311 raft_consensus.cc:2454] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1efc904c7764a46bb08655f1d54971d in term 0.
I20260809 00:42:21.137267 1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6658d95bdd44ebb9126133d8615b279" candidate_uuid: "e1efc904c7764a46bb08655f1d54971d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19e55899e2c4455aaab62be315b0564" is_pre_election: true
I20260809 00:42:21.137372 1518 raft_consensus.cc:2454] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1efc904c7764a46bb08655f1d54971d in term 0.
I20260809 00:42:21.137346 1622 leader_election.cc:302] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3c656803684b3c85151645dc488adf, e1efc904c7764a46bb08655f1d54971d; no voters:
I20260809 00:42:21.137488 1821 raft_consensus.cc:2781] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:21.137542 1821 raft_consensus.cc:482] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:21.137565 1821 raft_consensus.cc:3037] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:21.138286 1821 raft_consensus.cc:504] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.138444 1821 leader_election.cc:288] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [CANDIDATE]: Term 1 election: Requested vote from peers ce3c656803684b3c85151645dc488adf (127.30.104.193:44535), e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745)
I20260809 00:42:21.138618 1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6658d95bdd44ebb9126133d8615b279" candidate_uuid: "e1efc904c7764a46bb08655f1d54971d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19e55899e2c4455aaab62be315b0564"
I20260809 00:42:21.138610 1311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6658d95bdd44ebb9126133d8615b279" candidate_uuid: "e1efc904c7764a46bb08655f1d54971d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce3c656803684b3c85151645dc488adf"
I20260809 00:42:21.138696 1311 raft_consensus.cc:3037] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:21.138715 1518 raft_consensus.cc:3037] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:21.139358 1518 raft_consensus.cc:2454] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1efc904c7764a46bb08655f1d54971d in term 1.
I20260809 00:42:21.139416 1311 raft_consensus.cc:2454] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1efc904c7764a46bb08655f1d54971d in term 1.
I20260809 00:42:21.139561 1620 leader_election.cc:302] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e19e55899e2c4455aaab62be315b0564, e1efc904c7764a46bb08655f1d54971d; no voters:
I20260809 00:42:21.139671 1821 raft_consensus.cc:2781] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:21.139844 1821 raft_consensus.cc:686] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [term 1 LEADER]: Becoming Leader. State: Replica: e1efc904c7764a46bb08655f1d54971d, State: Running, Role: LEADER
I20260809 00:42:21.139959 1821 consensus_queue.cc:237] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.140405 1848 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: TxnStatusTablet state changed. Reason: New leader e1efc904c7764a46bb08655f1d54971d. Latest consensus state: current_term: 1 leader_uuid: "e1efc904c7764a46bb08655f1d54971d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:21.140496 1848 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:21.140744 1095 catalog_manager.cc:5495] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d reported cstate change: term changed from 0 to 1, leader changed from <none> to e1efc904c7764a46bb08655f1d54971d (127.30.104.195). New cstate: current_term: 1 leader_uuid: "e1efc904c7764a46bb08655f1d54971d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } health_report { overall_health: HEALTHY } } }
I20260809 00:42:21.141096 1852 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:42:21.141222 1852 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:42:21.168430 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.165632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49848 (local address 127.30.104.195:34399)
0809 00:42:21.165704 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.165706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.165716 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:21.165763 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.165765 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.165788 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.165868 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:21.166038 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.166601 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.167490 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.167720 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.167811 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.168030 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.168159 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.168261 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.168311 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.168339 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:42:21.168444 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.165371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:34399 (local address 127.0.0.1:49848)
0809 00:42:21.165443 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.165464 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.165521 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.165878 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.165879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.165881 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:21.165941 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.165944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.166703 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.166704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.167348 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.167349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.167809 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.167810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.167887 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.167916 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.168246 (+ 330us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.168257 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.168309 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.168366 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 00:42:21.170426 1311 raft_consensus.cc:1261] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Refusing update from remote peer e1efc904c7764a46bb08655f1d54971d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:21.170426 1518 raft_consensus.cc:1261] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Refusing update from remote peer e1efc904c7764a46bb08655f1d54971d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:21.170800 1848 consensus_queue.cc:1035] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:21.170960 1821 consensus_queue.cc:1035] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:21.172523 1819 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: TxnStatusTablet state changed. Reason: New leader e1efc904c7764a46bb08655f1d54971d. Latest consensus state: current_term: 1 leader_uuid: "e1efc904c7764a46bb08655f1d54971d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:21.172646 1819 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:21.172696 1825 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: TxnStatusTablet state changed. Reason: New leader e1efc904c7764a46bb08655f1d54971d. Latest consensus state: current_term: 1 leader_uuid: "e1efc904c7764a46bb08655f1d54971d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:21.172792 1825 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:21.173971 1821 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1efc904c7764a46bb08655f1d54971d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:21.174065 1821 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:21.174114 1821 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e1efc904c7764a46bb08655f1d54971d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:21.174152 1821 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P e1efc904c7764a46bb08655f1d54971d: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:21.174134 1819 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1efc904c7764a46bb08655f1d54971d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:21.174225 1819 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P e19e55899e2c4455aaab62be315b0564: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:21.174587 1825 tablet_replica.cc:439] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e1efc904c7764a46bb08655f1d54971d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } } }
I20260809 00:42:21.174667 1825 tablet_replica.cc:442] T c6658d95bdd44ebb9126133d8615b279 P ce3c656803684b3c85151645dc488adf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:21.174789 1861 mvcc.cc:205] Tried to move back new op lower bound from 7316423234231173120 to 7316423234109874176. Current Snapshot: MvccSnapshot[applied={T|T < 7316423234231173120}]
I20260809 00:42:21.175181 1862 mvcc.cc:205] Tried to move back new op lower bound from 7316423234231173120 to 7316423234109874176. Current Snapshot: MvccSnapshot[applied={T|T < 7316423234231173120 or (T in {7316423234231173120})}]
I20260809 00:42:21.179955 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.177060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43928 (local address 127.30.104.193:44535)
0809 00:42:21.177109 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.177112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.177255 (+ 143us) server_negotiation.cc:407] Connection header received
0809 00:42:21.177300 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.177301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.177318 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.177388 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:21.177537 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.178029 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.179136 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.179330 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.179390 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.179614 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.179720 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.179783 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.179852 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.179871 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:42:21.185583 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.182693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:41409 (local address 127.30.104.195:34399)
0809 00:42:21.182761 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.182763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.182885 (+ 122us) server_negotiation.cc:407] Connection header received
0809 00:42:21.182948 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.182949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.182960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.183011 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:21.183192 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.183732 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.184869 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.185074 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.185133 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.185346 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.185374 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.185414 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.185497 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.185518 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:42:21.185750 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.182633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:34399 (local address 127.30.104.193:41409)
0809 00:42:21.182810 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.182822 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.182887 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.183032 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.183033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.183035 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:21.183098 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.183100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.183807 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.183807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.184795 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.184798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.185154 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.185155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.185223 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.185240 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.185436 (+ 196us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.185438 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.185493 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.185539 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260809 00:42:21.189399 1712 raft_consensus.cc:1261] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Refusing update from remote peer ce3c656803684b3c85151645dc488adf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:21.189386 1518 raft_consensus.cc:1261] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Refusing update from remote peer ce3c656803684b3c85151645dc488adf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:21.189682 1829 consensus_queue.cc:1035] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:21.189828 1825 consensus_queue.cc:1035] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:21.192086 1860 mvcc.cc:205] Tried to move back new op lower bound from 7316423234309419008 to 7316423233129791488. Current Snapshot: MvccSnapshot[applied={T|T < 7316423234309419008}]
I20260809 00:42:21.199852 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.196669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:34475 (local address 127.30.104.193:44535)
0809 00:42:21.196729 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.196731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.196895 (+ 164us) server_negotiation.cc:407] Connection header received
0809 00:42:21.196961 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.196962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.196973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.197030 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:21.197285 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.197764 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.198935 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.199251 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.199293 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.199598 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.199629 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.199704 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.199775 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.199795 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:42:21.200202 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.196503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:44535 (local address 127.30.104.195:34475)
0809 00:42:21.196833 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.196844 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.196907 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.197038 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.197039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.197040 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:21.197199 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.197201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.197846 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.197847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.198694 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.198698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.199402 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.199402 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.199502 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.199520 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.199721 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.199723 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.199773 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.199856 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":75,"threads_started":1}
I20260809 00:42:21.210106 1095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:21.210263 1095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:21.211355 1095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: Sending DeleteTablet for 3 replicas of tablet 30af236e869c4a7cb9f455da1b53647c
I20260809 00:42:21.211802 1284 tablet_service.cc:1514] Processing DeleteTablet for tablet 30af236e869c4a7cb9f455da1b53647c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:21 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260809 00:42:21.211869 1488 tablet_service.cc:1514] Processing DeleteTablet for tablet 30af236e869c4a7cb9f455da1b53647c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:21 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38330
I20260809 00:42:21.212085 1689 tablet_service.cc:1514] Processing DeleteTablet for tablet 30af236e869c4a7cb9f455da1b53647c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:21 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:49836
I20260809 00:42:21.212312 1887 tablet_replica.cc:330] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: stopping tablet replica
I20260809 00:42:21.212468 1887 raft_consensus.cc:2229] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:21.212615 1887 raft_consensus.cc:2258] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:21.212589 1095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:21.212764 1095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:21.214290 1284 tablet_service.cc:1467] Processing CreateTablet for tablet 49e5c6fc529b489b9fd7fbd48166494d (DEFAULT_TABLE table=test-table [id=8e672791c32147c6935365d74824e247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:21.214401 1488 tablet_service.cc:1467] Processing CreateTablet for tablet 49e5c6fc529b489b9fd7fbd48166494d (DEFAULT_TABLE table=test-table [id=8e672791c32147c6935365d74824e247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:21.214459 1284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e5c6fc529b489b9fd7fbd48166494d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:21.214512 1488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e5c6fc529b489b9fd7fbd48166494d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:21.215514 1689 tablet_service.cc:1467] Processing CreateTablet for tablet 49e5c6fc529b489b9fd7fbd48166494d (DEFAULT_TABLE table=test-table [id=8e672791c32147c6935365d74824e247]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:21.215631 1689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e5c6fc529b489b9fd7fbd48166494d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:21.216037 1815 tablet_bootstrap.cc:492] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: Bootstrap starting.
I20260809 00:42:21.216495 1815 tablet_bootstrap.cc:654] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:21.217175 1812 tablet_bootstrap.cc:492] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: Bootstrap starting.
I20260809 00:42:21.217540 1812 tablet_bootstrap.cc:654] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:21.218505 1812 tablet_bootstrap.cc:492] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: No bootstrap required, opened a new log
I20260809 00:42:21.218562 1812 ts_tablet_manager.cc:1397] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:21.218700 1812 raft_consensus.cc:348] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.218796 1812 raft_consensus.cc:374] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:21.218824 1812 raft_consensus.cc:729] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1efc904c7764a46bb08655f1d54971d, State: Initialized, Role: FOLLOWER
I20260809 00:42:21.218873 1812 consensus_queue.cc:260] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.218972 1812 ts_tablet_manager.cc:1428] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:21.219875 1814 tablet_bootstrap.cc:492] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: Bootstrap starting.
I20260809 00:42:21.220307 1814 tablet_bootstrap.cc:654] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:21.220305 1887 ts_tablet_manager.cc:1905] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:21.221871 1886 tablet_replica.cc:330] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: stopping tablet replica
I20260809 00:42:21.222010 1886 raft_consensus.cc:2229] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:21.222090 1886 raft_consensus.cc:2258] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:21.222025 1887 ts_tablet_manager.cc:1918] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:21.222231 1887 log.cc:1196] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/30af236e869c4a7cb9f455da1b53647c
I20260809 00:42:21.222522 1886 ts_tablet_manager.cc:1905] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:21.222705 1887 ts_tablet_manager.cc:1939] T 30af236e869c4a7cb9f455da1b53647c P e1efc904c7764a46bb08655f1d54971d: Deleting consensus metadata
I20260809 00:42:21.223773 1074 catalog_manager.cc:4841] TS e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399): tablet 30af236e869c4a7cb9f455da1b53647c (table test-table [id=5034d4f682be48b79f26dfa45f731f6d]) successfully deleted
I20260809 00:42:21.223897 1815 tablet_bootstrap.cc:492] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: No bootstrap required, opened a new log
I20260809 00:42:21.223963 1815 ts_tablet_manager.cc:1397] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260809 00:42:21.224136 1815 raft_consensus.cc:348] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.224221 1815 raft_consensus.cc:374] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:21.224242 1815 raft_consensus.cc:729] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3c656803684b3c85151645dc488adf, State: Initialized, Role: FOLLOWER
I20260809 00:42:21.224285 1815 consensus_queue.cc:260] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.224399 1815 ts_tablet_manager.cc:1428] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:21.224411 1886 ts_tablet_manager.cc:1918] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:21.224484 1886 log.cc:1196] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/30af236e869c4a7cb9f455da1b53647c
I20260809 00:42:21.224767 1886 ts_tablet_manager.cc:1939] T 30af236e869c4a7cb9f455da1b53647c P e19e55899e2c4455aaab62be315b0564: Deleting consensus metadata
I20260809 00:42:21.225219 1885 tablet_replica.cc:330] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: stopping tablet replica
I20260809 00:42:21.225221 1074 catalog_manager.cc:4841] TS e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745): tablet 30af236e869c4a7cb9f455da1b53647c (table test-table [id=5034d4f682be48b79f26dfa45f731f6d]) successfully deleted
I20260809 00:42:21.225345 1885 raft_consensus.cc:2229] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:21.225458 1885 raft_consensus.cc:2258] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:21.225857 1885 ts_tablet_manager.cc:1905] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:21.227473 1885 ts_tablet_manager.cc:1918] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:21.227551 1885 log.cc:1196] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/30af236e869c4a7cb9f455da1b53647c
I20260809 00:42:21.227799 1885 ts_tablet_manager.cc:1939] T 30af236e869c4a7cb9f455da1b53647c P ce3c656803684b3c85151645dc488adf: Deleting consensus metadata
I20260809 00:42:21.228214 1078 catalog_manager.cc:4841] TS ce3c656803684b3c85151645dc488adf (127.30.104.193:44535): tablet 30af236e869c4a7cb9f455da1b53647c (table test-table [id=5034d4f682be48b79f26dfa45f731f6d]) successfully deleted
I20260809 00:42:21.232208 1814 tablet_bootstrap.cc:492] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: No bootstrap required, opened a new log
I20260809 00:42:21.232306 1814 ts_tablet_manager.cc:1397] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260809 00:42:21.232492 1814 raft_consensus.cc:348] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.232585 1814 raft_consensus.cc:374] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:21.232614 1814 raft_consensus.cc:729] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19e55899e2c4455aaab62be315b0564, State: Initialized, Role: FOLLOWER
I20260809 00:42:21.232663 1814 consensus_queue.cc:260] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.232767 1814 ts_tablet_manager.cc:1428] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:21.255143 1819 raft_consensus.cc:482] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:21.255275 1819 raft_consensus.cc:504] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.255733 1819 leader_election.cc:288] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce3c656803684b3c85151645dc488adf (127.30.104.193:44535), e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:21.260375 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.256112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:55337 (local address 127.30.104.195:34399)
0809 00:42:21.256155 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.256157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.256316 (+ 159us) server_negotiation.cc:407] Connection header received
0809 00:42:21.256434 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.256436 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.256449 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.256485 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:21.256764 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.257520 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.259222 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.259828 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.259869 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.259876 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.260242 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.260298 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:42:21.260375 1810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.256098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:51385 (local address 127.30.104.193:44535)
0809 00:42:21.256146 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.256148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.256288 (+ 140us) server_negotiation.cc:407] Connection header received
0809 00:42:21.256406 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.256407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.256419 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.256453 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:21.256764 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.257512 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.259222 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.259807 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.259866 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.259876 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.260242 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.260293 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:42:21.260725 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.255890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:44535 (local address 127.30.104.194:51385)
0809 00:42:21.256210 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.256223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.256319 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.256538 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.256539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.256541 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:21.256664 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.256668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.257623 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.257624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.259092 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.259097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.260038 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.260038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.260135 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.260137 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.260567 (+ 430us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.260634 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":140,"thread_start_us":83,"threads_started":1}
I20260809 00:42:21.261000 1311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e5c6fc529b489b9fd7fbd48166494d" candidate_uuid: "e19e55899e2c4455aaab62be315b0564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce3c656803684b3c85151645dc488adf" is_pre_election: true
I20260809 00:42:21.261114 1311 raft_consensus.cc:2454] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19e55899e2c4455aaab62be315b0564 in term 0.
I20260809 00:42:21.261323 1422 leader_election.cc:302] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3c656803684b3c85151645dc488adf, e19e55899e2c4455aaab62be315b0564; no voters:
I20260809 00:42:21.261511 1819 raft_consensus.cc:2781] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:21.261554 1819 raft_consensus.cc:482] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:21.261580 1819 raft_consensus.cc:3037] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:21.262441 1819 raft_consensus.cc:504] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.262658 1819 leader_election.cc:288] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [CANDIDATE]: Term 1 election: Requested vote from peers ce3c656803684b3c85151645dc488adf (127.30.104.193:44535), e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:21.261309 1894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.255936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:34399 (local address 127.30.104.194:55337)
0809 00:42:21.256261 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.256270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.256340 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.256547 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.256547 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.256548 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:21.256664 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.256667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.257624 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.257624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.259092 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.259097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.260036 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.260037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.260132 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.260136 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.261053 (+ 917us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.261224 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":98,"thread_start_us":76,"threads_started":1}
I20260809 00:42:21.262907 1311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e5c6fc529b489b9fd7fbd48166494d" candidate_uuid: "e19e55899e2c4455aaab62be315b0564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce3c656803684b3c85151645dc488adf"
I20260809 00:42:21.262995 1311 raft_consensus.cc:3037] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:21.263141 1712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e5c6fc529b489b9fd7fbd48166494d" candidate_uuid: "e19e55899e2c4455aaab62be315b0564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1efc904c7764a46bb08655f1d54971d" is_pre_election: true
I20260809 00:42:21.263252 1712 raft_consensus.cc:2454] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19e55899e2c4455aaab62be315b0564 in term 0.
I20260809 00:42:21.263190 1714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e5c6fc529b489b9fd7fbd48166494d" candidate_uuid: "e19e55899e2c4455aaab62be315b0564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1efc904c7764a46bb08655f1d54971d"
I20260809 00:42:21.263310 1714 raft_consensus.cc:3037] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:21.263931 1311 raft_consensus.cc:2454] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19e55899e2c4455aaab62be315b0564 in term 1.
I20260809 00:42:21.264226 1714 raft_consensus.cc:2454] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19e55899e2c4455aaab62be315b0564 in term 1.
I20260809 00:42:21.264225 1422 leader_election.cc:302] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3c656803684b3c85151645dc488adf, e19e55899e2c4455aaab62be315b0564; no voters:
I20260809 00:42:21.264525 1819 raft_consensus.cc:2781] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:21.264760 1819 raft_consensus.cc:686] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 1 LEADER]: Becoming Leader. State: Replica: e19e55899e2c4455aaab62be315b0564, State: Running, Role: LEADER
I20260809 00:42:21.264845 1819 consensus_queue.cc:237] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:21.265756 1095 catalog_manager.cc:5495] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 reported cstate change: term changed from 0 to 1, leader changed from <none> to e19e55899e2c4455aaab62be315b0564 (127.30.104.194). New cstate: current_term: 1 leader_uuid: "e19e55899e2c4455aaab62be315b0564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:21.283231 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.280658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38338 (local address 127.30.104.194:36745)
0809 00:42:21.280698 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.280701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.280739 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:42:21.280825 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.280826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.280851 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.280955 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:21.281089 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.281577 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.282326 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.282542 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.282625 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.282813 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.282957 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.283047 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.283104 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.283129 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:21.288479 1899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.285834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:34399 (local address 127.30.104.194:55687)
0809 00:42:21.285996 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.286007 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.286068 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.286191 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.286192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.286194 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:21.286262 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.286263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.286905 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.286907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.287612 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.287615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.287998 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.287999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.288068 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.288090 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.288288 (+ 198us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.288291 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.288344 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.288397 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260809 00:42:21.288457 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.285889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:55687 (local address 127.30.104.195:34399)
0809 00:42:21.285992 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.285994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.286052 (+ 58us) server_negotiation.cc:407] Connection header received
0809 00:42:21.286120 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.286121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.286132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.286189 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:21.286338 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.286811 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.287703 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.287912 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.287995 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.288176 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.288224 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.288287 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.288351 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.288374 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 00:42:21.291641 1311 raft_consensus.cc:1261] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Refusing update from remote peer e19e55899e2c4455aaab62be315b0564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:21.291700 1714 raft_consensus.cc:1261] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Refusing update from remote peer e19e55899e2c4455aaab62be315b0564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:21.291949 1819 consensus_queue.cc:1035] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:21.292073 1895 consensus_queue.cc:1035] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:21.301916 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.298331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:36745 (local address 127.30.104.195:40761)
0809 00:42:21.298401 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:21.298413 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:21.298454 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:21.298754 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:21.298755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:21.298757 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:21.298807 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.298808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.299679 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.299680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.300682 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:21.300684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.301532 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.301533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:21.301585 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.301592 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.301739 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.301741 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:42:21.301774 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 00:42:21.301817 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 00:42:21.302106 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:21.298573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:40761 (local address 127.30.104.194:36745)
0809 00:42:21.298613 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:21.298615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:21.298622 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:42:21.298644 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:21.298644 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:21.298655 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:21.298702 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:21.298879 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.299598 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.301178 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:21.301443 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:21.301499 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:21.301654 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:21.301687 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:21.301723 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:21.302029 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 00:42:21.302047 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:42:21.308136 1862 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 00:42:22.840317 1096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:22.840454 1096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:22.841545 1096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: Sending DeleteTablet for 3 replicas of tablet 49e5c6fc529b489b9fd7fbd48166494d
I20260809 00:42:22.841919 1284 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e5c6fc529b489b9fd7fbd48166494d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260809 00:42:22.841940 1689 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e5c6fc529b489b9fd7fbd48166494d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:49836
I20260809 00:42:22.842182 1935 tablet_replica.cc:330] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: stopping tablet replica
I20260809 00:42:22.842247 1935 raft_consensus.cc:2229] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:22.842231 1488 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e5c6fc529b489b9fd7fbd48166494d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38330
I20260809 00:42:22.842298 1935 raft_consensus.cc:2258] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:22.842435 1936 tablet_replica.cc:330] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: stopping tablet replica
I20260809 00:42:22.842496 1936 raft_consensus.cc:2229] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:22.842592 1936 raft_consensus.cc:2258] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:22.842859 1936 ts_tablet_manager.cc:1905] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:22.844095 1935 ts_tablet_manager.cc:1905] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:22.846271 1935 ts_tablet_manager.cc:1918] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:42:22.846349 1935 log.cc:1196] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/49e5c6fc529b489b9fd7fbd48166494d
I20260809 00:42:22.846535 1934 tablet_replica.cc:330] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: stopping tablet replica
I20260809 00:42:22.846573 1935 ts_tablet_manager.cc:1939] T 49e5c6fc529b489b9fd7fbd48166494d P e1efc904c7764a46bb08655f1d54971d: Deleting consensus metadata
I20260809 00:42:22.846660 1934 raft_consensus.cc:2229] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:22.846848 1934 raft_consensus.cc:2258] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:22.846973 1074 catalog_manager.cc:4841] TS e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399): tablet 49e5c6fc529b489b9fd7fbd48166494d (table test-table [id=8e672791c32147c6935365d74824e247]) successfully deleted
I20260809 00:42:22.847229 1934 ts_tablet_manager.cc:1905] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:22.848470 1936 ts_tablet_manager.cc:1918] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:42:22.848551 1936 log.cc:1196] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/49e5c6fc529b489b9fd7fbd48166494d
I20260809 00:42:22.848819 1936 ts_tablet_manager.cc:1939] T 49e5c6fc529b489b9fd7fbd48166494d P e19e55899e2c4455aaab62be315b0564: Deleting consensus metadata
I20260809 00:42:22.849220 1074 catalog_manager.cc:4841] TS e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745): tablet 49e5c6fc529b489b9fd7fbd48166494d (table test-table [id=8e672791c32147c6935365d74824e247]) successfully deleted
I20260809 00:42:22.849306 1934 ts_tablet_manager.cc:1918] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:42:22.849371 1934 log.cc:1196] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/49e5c6fc529b489b9fd7fbd48166494d
I20260809 00:42:22.849575 1934 ts_tablet_manager.cc:1939] T 49e5c6fc529b489b9fd7fbd48166494d P ce3c656803684b3c85151645dc488adf: Deleting consensus metadata
I20260809 00:42:22.849936 1078 catalog_manager.cc:4841] TS ce3c656803684b3c85151645dc488adf (127.30.104.193:44535): tablet 49e5c6fc529b489b9fd7fbd48166494d (table test-table [id=8e672791c32147c6935365d74824e247]) successfully deleted
Aug 09 00:42:22 dist-test-slave-c12x krb5kdc[1039](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:42:22 dist-test-slave-c12x krb5kdc[1039](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236142, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:22.876063 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:22.861618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.30.104.254:33225)
0809 00:42:22.861761 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:22.861763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:22.861773 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:22.861903 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:22.861904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:22.861915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:22.861989 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:22.862674 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:22.863301 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:22.864184 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:22.864400 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:22.864462 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:22.873100 (+ 8638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:22.873107 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:22.873108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:22.873116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:22.874607 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:22.875071 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:22.875072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:22.875073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:22.875149 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:22.875369 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:22.875370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:22.875370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:22.875579 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:22.875698 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:22.875843 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:42:22.875944 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260809 00:42:22.877856 1096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:22.878159 1096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:22.879916 1488 tablet_service.cc:1467] Processing CreateTablet for tablet 5dd93a2760724e938931d27fb2d8e9d2 (DEFAULT_TABLE table=test-table [id=0665321a6cb942369c03373ada3787d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:22.879920 1284 tablet_service.cc:1467] Processing CreateTablet for tablet 5dd93a2760724e938931d27fb2d8e9d2 (DEFAULT_TABLE table=test-table [id=0665321a6cb942369c03373ada3787d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:22.879938 1689 tablet_service.cc:1467] Processing CreateTablet for tablet 5dd93a2760724e938931d27fb2d8e9d2 (DEFAULT_TABLE table=test-table [id=0665321a6cb942369c03373ada3787d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:22.880106 1689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dd93a2760724e938931d27fb2d8e9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:22.880106 1488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dd93a2760724e938931d27fb2d8e9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:22.880110 1284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dd93a2760724e938931d27fb2d8e9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:22.881408 1948 tablet_bootstrap.cc:492] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: Bootstrap starting.
I20260809 00:42:22.881675 1950 tablet_bootstrap.cc:492] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: Bootstrap starting.
I20260809 00:42:22.881884 1948 tablet_bootstrap.cc:654] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:22.881916 1949 tablet_bootstrap.cc:492] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: Bootstrap starting.
I20260809 00:42:22.882077 1950 tablet_bootstrap.cc:654] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:22.882287 1949 tablet_bootstrap.cc:654] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:22.882673 1948 tablet_bootstrap.cc:492] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: No bootstrap required, opened a new log
I20260809 00:42:22.882746 1948 ts_tablet_manager.cc:1397] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:22.882920 1948 raft_consensus.cc:348] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.883020 1949 tablet_bootstrap.cc:492] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: No bootstrap required, opened a new log
I20260809 00:42:22.883021 1948 raft_consensus.cc:374] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:22.883052 1948 raft_consensus.cc:729] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3c656803684b3c85151645dc488adf, State: Initialized, Role: FOLLOWER
I20260809 00:42:22.883086 1949 ts_tablet_manager.cc:1397] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:22.883118 1948 consensus_queue.cc:260] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.883256 1949 raft_consensus.cc:348] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.883311 1948 ts_tablet_manager.cc:1428] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:22.883353 1949 raft_consensus.cc:374] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:22.883376 1949 raft_consensus.cc:729] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1efc904c7764a46bb08655f1d54971d, State: Initialized, Role: FOLLOWER
I20260809 00:42:22.883481 1949 consensus_queue.cc:260] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.883870 1949 ts_tablet_manager.cc:1428] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:22.884045 1950 tablet_bootstrap.cc:492] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: No bootstrap required, opened a new log
I20260809 00:42:22.884119 1950 ts_tablet_manager.cc:1397] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:22.884245 1950 raft_consensus.cc:348] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.884331 1950 raft_consensus.cc:374] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:22.884361 1950 raft_consensus.cc:729] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19e55899e2c4455aaab62be315b0564, State: Initialized, Role: FOLLOWER
I20260809 00:42:22.884438 1950 consensus_queue.cc:260] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.884550 1950 ts_tablet_manager.cc:1428] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:22.949260 1963 raft_consensus.cc:482] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:22.949373 1963 raft_consensus.cc:504] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.949597 1963 leader_election.cc:288] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745), e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:22.949926 1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dd93a2760724e938931d27fb2d8e9d2" candidate_uuid: "ce3c656803684b3c85151645dc488adf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19e55899e2c4455aaab62be315b0564" is_pre_election: true
I20260809 00:42:22.950052 1518 raft_consensus.cc:2454] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce3c656803684b3c85151645dc488adf in term 0.
I20260809 00:42:22.949985 1714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dd93a2760724e938931d27fb2d8e9d2" candidate_uuid: "ce3c656803684b3c85151645dc488adf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1efc904c7764a46bb08655f1d54971d" is_pre_election: true
I20260809 00:42:22.950121 1714 raft_consensus.cc:2454] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce3c656803684b3c85151645dc488adf in term 0.
I20260809 00:42:22.950249 1215 leader_election.cc:302] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3c656803684b3c85151645dc488adf, e19e55899e2c4455aaab62be315b0564; no voters:
I20260809 00:42:22.950377 1963 raft_consensus.cc:2781] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:22.950440 1963 raft_consensus.cc:482] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:22.950457 1963 raft_consensus.cc:3037] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:22.951228 1963 raft_consensus.cc:504] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.951336 1963 leader_election.cc:288] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [CANDIDATE]: Term 1 election: Requested vote from peers e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745), e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:22.951608 1518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dd93a2760724e938931d27fb2d8e9d2" candidate_uuid: "ce3c656803684b3c85151645dc488adf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e19e55899e2c4455aaab62be315b0564"
I20260809 00:42:22.951704 1518 raft_consensus.cc:3037] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:22.951671 1714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dd93a2760724e938931d27fb2d8e9d2" candidate_uuid: "ce3c656803684b3c85151645dc488adf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1efc904c7764a46bb08655f1d54971d"
I20260809 00:42:22.951792 1714 raft_consensus.cc:3037] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:22.952353 1518 raft_consensus.cc:2454] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce3c656803684b3c85151645dc488adf in term 1.
I20260809 00:42:22.952508 1714 raft_consensus.cc:2454] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce3c656803684b3c85151645dc488adf in term 1.
I20260809 00:42:22.952544 1215 leader_election.cc:302] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce3c656803684b3c85151645dc488adf, e19e55899e2c4455aaab62be315b0564; no voters:
I20260809 00:42:22.952627 1963 raft_consensus.cc:2781] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:22.952822 1963 raft_consensus.cc:686] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 1 LEADER]: Becoming Leader. State: Replica: ce3c656803684b3c85151645dc488adf, State: Running, Role: LEADER
I20260809 00:42:22.952901 1963 consensus_queue.cc:237] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.953611 1094 catalog_manager.cc:5495] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf reported cstate change: term changed from 0 to 1, leader changed from <none> to ce3c656803684b3c85151645dc488adf (127.30.104.193). New cstate: current_term: 1 leader_uuid: "ce3c656803684b3c85151645dc488adf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:22.962826 1968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:22.959862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43932 (local address 127.30.104.193:44535)
0809 00:42:22.960059 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:22.960061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:22.960069 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:22.960097 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:22.960098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:22.960107 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:22.960168 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:22.960329 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:22.960814 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:22.961821 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:22.962049 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:22.962113 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:22.962413 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:22.962448 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:22.962511 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:22.962647 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 00:42:22.962679 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":114,"threads_started":1}
I20260809 00:42:22.965649 1518 raft_consensus.cc:1261] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Refusing update from remote peer ce3c656803684b3c85151645dc488adf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:22.965701 1714 raft_consensus.cc:1261] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Refusing update from remote peer ce3c656803684b3c85151645dc488adf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:22.965880 1964 consensus_queue.cc:1035] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:22.966056 1963 consensus_queue.cc:1035] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:22.978058 1094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:22.978168 1094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:22.978937 1094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: Sending DeleteTablet for 3 replicas of tablet 5dd93a2760724e938931d27fb2d8e9d2
I20260809 00:42:22.979216 1284 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dd93a2760724e938931d27fb2d8e9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260809 00:42:22.979238 1689 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dd93a2760724e938931d27fb2d8e9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:49836
I20260809 00:42:22.979252 1488 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dd93a2760724e938931d27fb2d8e9d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38330
I20260809 00:42:22.979328 1934 tablet_replica.cc:330] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: stopping tablet replica
I20260809 00:42:22.979359 1935 tablet_replica.cc:330] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: stopping tablet replica
I20260809 00:42:22.979385 1934 raft_consensus.cc:2229] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:22.979473 1934 raft_consensus.cc:2258] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:22.979487 1935 raft_consensus.cc:2229] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:22.979540 1935 raft_consensus.cc:2258] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:22.979421 1094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:22.979611 1094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:22.979621 1936 tablet_replica.cc:330] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: stopping tablet replica
I20260809 00:42:22.979676 1936 raft_consensus.cc:2229] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:22.979714 1936 raft_consensus.cc:2258] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:22.979849 1934 ts_tablet_manager.cc:1905] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:22.980079 1936 ts_tablet_manager.cc:1905] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:22.981034 1488 tablet_service.cc:1467] Processing CreateTablet for tablet 5739d4a8715e42809ecce67ac1f40104 (DEFAULT_TABLE table=test-table [id=75280b04306c4e5e8450b3ecf1f5b14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:22.981153 1488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5739d4a8715e42809ecce67ac1f40104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:22.981361 1935 ts_tablet_manager.cc:1905] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:22.981555 1934 ts_tablet_manager.cc:1918] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:22.981607 1934 log.cc:1196] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/5dd93a2760724e938931d27fb2d8e9d2
I20260809 00:42:22.981829 1934 ts_tablet_manager.cc:1939] T 5dd93a2760724e938931d27fb2d8e9d2 P ce3c656803684b3c85151645dc488adf: Deleting consensus metadata
I20260809 00:42:22.982084 1936 ts_tablet_manager.cc:1918] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:22.982137 1936 log.cc:1196] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/5dd93a2760724e938931d27fb2d8e9d2
I20260809 00:42:22.982317 1936 ts_tablet_manager.cc:1939] T 5dd93a2760724e938931d27fb2d8e9d2 P e19e55899e2c4455aaab62be315b0564: Deleting consensus metadata
I20260809 00:42:22.982587 1950 tablet_bootstrap.cc:492] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: Bootstrap starting.
I20260809 00:42:22.982628 1074 catalog_manager.cc:4841] TS e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745): tablet 5dd93a2760724e938931d27fb2d8e9d2 (table test-table [id=0665321a6cb942369c03373ada3787d9]) successfully deleted
I20260809 00:42:22.982921 1950 tablet_bootstrap.cc:654] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:22.982933 1935 ts_tablet_manager.cc:1918] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:22.982985 1935 log.cc:1196] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/5dd93a2760724e938931d27fb2d8e9d2
I20260809 00:42:22.983206 1935 ts_tablet_manager.cc:1939] T 5dd93a2760724e938931d27fb2d8e9d2 P e1efc904c7764a46bb08655f1d54971d: Deleting consensus metadata
I20260809 00:42:22.983237 1689 tablet_service.cc:1467] Processing CreateTablet for tablet 5739d4a8715e42809ecce67ac1f40104 (DEFAULT_TABLE table=test-table [id=75280b04306c4e5e8450b3ecf1f5b14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:22.983331 1689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5739d4a8715e42809ecce67ac1f40104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:22.983677 1950 tablet_bootstrap.cc:492] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: No bootstrap required, opened a new log
I20260809 00:42:22.983733 1950 ts_tablet_manager.cc:1397] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:22.983845 1950 raft_consensus.cc:348] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.983932 1950 raft_consensus.cc:374] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:22.983960 1950 raft_consensus.cc:729] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e19e55899e2c4455aaab62be315b0564, State: Initialized, Role: FOLLOWER
I20260809 00:42:22.984014 1950 consensus_queue.cc:260] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.984139 1950 ts_tablet_manager.cc:1428] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:22.984551 1949 tablet_bootstrap.cc:492] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: Bootstrap starting.
I20260809 00:42:22.984692 1074 catalog_manager.cc:4841] TS e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399): tablet 5dd93a2760724e938931d27fb2d8e9d2 (table test-table [id=0665321a6cb942369c03373ada3787d9]) successfully deleted
I20260809 00:42:22.984875 1949 tablet_bootstrap.cc:654] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:22.985023 1078 catalog_manager.cc:4841] TS ce3c656803684b3c85151645dc488adf (127.30.104.193:44535): tablet 5dd93a2760724e938931d27fb2d8e9d2 (table test-table [id=0665321a6cb942369c03373ada3787d9]) successfully deleted
I20260809 00:42:22.985204 1284 tablet_service.cc:1467] Processing CreateTablet for tablet 5739d4a8715e42809ecce67ac1f40104 (DEFAULT_TABLE table=test-table [id=75280b04306c4e5e8450b3ecf1f5b14e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:22.985327 1284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5739d4a8715e42809ecce67ac1f40104. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:22.986234 1948 tablet_bootstrap.cc:492] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf: Bootstrap starting.
I20260809 00:42:22.986393 1949 tablet_bootstrap.cc:492] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: No bootstrap required, opened a new log
I20260809 00:42:22.986454 1949 ts_tablet_manager.cc:1397] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:22.986572 1949 raft_consensus.cc:348] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.986644 1949 raft_consensus.cc:374] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:22.986658 1949 raft_consensus.cc:729] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1efc904c7764a46bb08655f1d54971d, State: Initialized, Role: FOLLOWER
I20260809 00:42:22.986627 1948 tablet_bootstrap.cc:654] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:22.986711 1949 consensus_queue.cc:260] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.986819 1949 ts_tablet_manager.cc:1428] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:22.987478 1948 tablet_bootstrap.cc:492] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf: No bootstrap required, opened a new log
I20260809 00:42:22.987550 1948 ts_tablet_manager.cc:1397] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:22.987668 1948 raft_consensus.cc:348] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.987769 1948 raft_consensus.cc:374] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:22.987802 1948 raft_consensus.cc:729] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce3c656803684b3c85151645dc488adf, State: Initialized, Role: FOLLOWER
I20260809 00:42:22.987860 1948 consensus_queue.cc:260] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:22.987985 1948 ts_tablet_manager.cc:1428] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:23.009140 1922 raft_consensus.cc:482] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:23.009274 1922 raft_consensus.cc:504] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:23.009488 1922 leader_election.cc:288] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce3c656803684b3c85151645dc488adf (127.30.104.193:44535), e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:23.009829 1714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5739d4a8715e42809ecce67ac1f40104" candidate_uuid: "e19e55899e2c4455aaab62be315b0564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1efc904c7764a46bb08655f1d54971d" is_pre_election: true
I20260809 00:42:23.009817 1309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5739d4a8715e42809ecce67ac1f40104" candidate_uuid: "e19e55899e2c4455aaab62be315b0564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce3c656803684b3c85151645dc488adf" is_pre_election: true
I20260809 00:42:23.009930 1714 raft_consensus.cc:2454] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19e55899e2c4455aaab62be315b0564 in term 0.
I20260809 00:42:23.009960 1309 raft_consensus.cc:2454] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e19e55899e2c4455aaab62be315b0564 in term 0.
I20260809 00:42:23.010155 1419 leader_election.cc:302] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e19e55899e2c4455aaab62be315b0564, e1efc904c7764a46bb08655f1d54971d; no voters:
I20260809 00:42:23.010267 1922 raft_consensus.cc:2781] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:23.010309 1922 raft_consensus.cc:482] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:23.010339 1922 raft_consensus.cc:3037] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:23.011101 1922 raft_consensus.cc:504] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:23.011251 1922 leader_election.cc:288] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [CANDIDATE]: Term 1 election: Requested vote from peers ce3c656803684b3c85151645dc488adf (127.30.104.193:44535), e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399)
I20260809 00:42:23.011409 1714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5739d4a8715e42809ecce67ac1f40104" candidate_uuid: "e19e55899e2c4455aaab62be315b0564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1efc904c7764a46bb08655f1d54971d"
I20260809 00:42:23.011579 1714 raft_consensus.cc:3037] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:23.011514 1309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5739d4a8715e42809ecce67ac1f40104" candidate_uuid: "e19e55899e2c4455aaab62be315b0564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce3c656803684b3c85151645dc488adf"
I20260809 00:42:23.011631 1309 raft_consensus.cc:3037] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:23.012241 1714 raft_consensus.cc:2454] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19e55899e2c4455aaab62be315b0564 in term 1.
I20260809 00:42:23.012431 1419 leader_election.cc:302] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e19e55899e2c4455aaab62be315b0564, e1efc904c7764a46bb08655f1d54971d; no voters:
I20260809 00:42:23.012553 1922 raft_consensus.cc:2781] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:23.012562 1309 raft_consensus.cc:2454] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e19e55899e2c4455aaab62be315b0564 in term 1.
I20260809 00:42:23.012607 1922 raft_consensus.cc:686] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 1 LEADER]: Becoming Leader. State: Replica: e19e55899e2c4455aaab62be315b0564, State: Running, Role: LEADER
I20260809 00:42:23.012673 1922 consensus_queue.cc:237] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } }
I20260809 00:42:23.013387 1094 catalog_manager.cc:5495] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 reported cstate change: term changed from 0 to 1, leader changed from <none> to e19e55899e2c4455aaab62be315b0564 (127.30.104.194). New cstate: current_term: 1 leader_uuid: "e19e55899e2c4455aaab62be315b0564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e19e55899e2c4455aaab62be315b0564" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36745 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:23.021135 1983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:23.018462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38348 (local address 127.30.104.194:36745)
0809 00:42:23.018702 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:23.018704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:23.018713 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:23.018733 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:23.018734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:23.018744 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:23.018809 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:23.018928 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:23.019528 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:23.020314 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:23.020545 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:23.020614 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:23.020825 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:23.020861 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:23.020917 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:23.020982 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 00:42:23.021002 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":150,"threads_started":1}
I20260809 00:42:23.023984 1714 raft_consensus.cc:1261] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Refusing update from remote peer e19e55899e2c4455aaab62be315b0564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:23.024045 1309 raft_consensus.cc:1261] T 5739d4a8715e42809ecce67ac1f40104 P ce3c656803684b3c85151645dc488adf [term 1 FOLLOWER]: Refusing update from remote peer e19e55899e2c4455aaab62be315b0564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:23.024245 1819 consensus_queue.cc:1035] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1efc904c7764a46bb08655f1d54971d" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:23.024353 1922 consensus_queue.cc:1035] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce3c656803684b3c85151645dc488adf" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 44535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:42:23.029654 1912 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 00:42:24.552690 1094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:24.552803 1094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:24.553824 1094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5157724a8b6d4a6fbecfbcd3b5b3e689: Sending DeleteTablet for 3 replicas of tablet 5739d4a8715e42809ecce67ac1f40104
I20260809 00:42:24.554101 1488 tablet_service.cc:1514] Processing DeleteTablet for tablet 5739d4a8715e42809ecce67ac1f40104 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:24 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38330
I20260809 00:42:24.554140 1689 tablet_service.cc:1514] Processing DeleteTablet for tablet 5739d4a8715e42809ecce67ac1f40104 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:24 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:49836
I20260809 00:42:24.554150 1284 tablet_service.cc:1514] Processing DeleteTablet for tablet 5739d4a8715e42809ecce67ac1f40104 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:24 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:43918
I20260809 00:42:24.554268 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 1194
I20260809 00:42:24.554468 2024 tablet_replica.cc:330] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: stopping tablet replica
I20260809 00:42:24.554581 2026 tablet_replica.cc:330] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: stopping tablet replica
I20260809 00:42:24.554692 2026 raft_consensus.cc:2229] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:24.558019 2026 raft_consensus.cc:2258] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:24.557952 2024 raft_consensus.cc:2229] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:24.558274 2024 raft_consensus.cc:2258] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:24.558955 2026 ts_tablet_manager.cc:1905] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:24.560508 2024 ts_tablet_manager.cc:1905] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:24.562083 2024 ts_tablet_manager.cc:1918] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:42:24.563984 2026 ts_tablet_manager.cc:1918] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 00:42:24.564069 2026 log.cc:1196] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/5739d4a8715e42809ecce67ac1f40104
I20260809 00:42:24.562178 2024 log.cc:1196] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/5739d4a8715e42809ecce67ac1f40104
I20260809 00:42:24.564678 2024 ts_tablet_manager.cc:1939] T 5739d4a8715e42809ecce67ac1f40104 P e19e55899e2c4455aaab62be315b0564: Deleting consensus metadata
I20260809 00:42:24.565205 1074 catalog_manager.cc:4841] TS e19e55899e2c4455aaab62be315b0564 (127.30.104.194:36745): tablet 5739d4a8715e42809ecce67ac1f40104 (table test-table [id=75280b04306c4e5e8450b3ecf1f5b14e]) successfully deleted
I20260809 00:42:24.565315 2026 ts_tablet_manager.cc:1939] T 5739d4a8715e42809ecce67ac1f40104 P e1efc904c7764a46bb08655f1d54971d: Deleting consensus metadata
I20260809 00:42:24.565825 1074 catalog_manager.cc:4841] TS e1efc904c7764a46bb08655f1d54971d (127.30.104.195:34399): tablet 5739d4a8715e42809ecce67ac1f40104 (table test-table [id=75280b04306c4e5e8450b3ecf1f5b14e]) successfully deleted
W20260809 00:42:24.574419 1078 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:44535 (error 108)
I20260809 00:42:24.574824 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 1398
I20260809 00:42:24.580044 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:24.575012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:44535 (local address 127.0.0.1:43940)
0809 00:42:24.579888 (+ 4876us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:24.579976 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:44535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4834,"thread_start_us":83,"threads_started":1}
W20260809 00:42:24.582425 1078 catalog_manager.cc:4584] TS ce3c656803684b3c85151645dc488adf (127.30.104.193:44535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5739d4a8715e42809ecce67ac1f40104: Network error: Client connection negotiation failed: client connection to 127.30.104.193:44535: connect: Connection refused (error 111)
I20260809 00:42:24.588966 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 1598
I20260809 00:42:24.603165 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 1053
2026-08-09T00:42:24Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6134 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:24 dist-test-slave-c12x krb5kdc[2034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:24 dist-test-slave-c12x krb5kdc[2034](info): set up 2 sockets
Aug 09 00:42:24 dist-test-slave-c12x krb5kdc[2034](info): commencing operation
krb5kdc: starting...
W20260809 00:42:25.842856 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:25 dist-test-slave-c12x krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:25.974610 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37841
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:26.063949 2055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:26.064230 2055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:26.064275 2055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:26.065507 2055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:26.065584 2055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:26.065609 2055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:26.065639 2055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:26.065666 2055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:26.067224 2055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:26.067556 2055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:26.067804 2055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:26.073354 2060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:26.073388 2063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:26.073410 2061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:26.073455 2055 server_base.cc:1034] running on GCE node
I20260809 00:42:26.074179 2055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:26.074483 2055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:26.075627 2055 hybrid_clock.cc:648] HybridClock initialized: now 1786236146075608 us; error 43 us; skew 500 ppm
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236146, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:26.078709 2055 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:26.079218 2055 webserver.cc:460] Webserver started at http://127.30.104.254:44883/ using document root <none> and password file <none>
I20260809 00:42:26.079523 2055 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:26.079577 2055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:26.079680 2055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:26.080819 2055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "5259b1842f6f65490221ce9f4ea4ee4a"
server_key_iv: "32953d84b3b8748692444413f5561f16"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.081169 2055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "5259b1842f6f65490221ce9f4ea4ee4a"
server_key_iv: "32953d84b3b8748692444413f5561f16"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.082676 2055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:26.083546 2075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.083712 2055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:26.083801 2055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "5259b1842f6f65490221ce9f4ea4ee4a"
server_key_iv: "32953d84b3b8748692444413f5561f16"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.083868 2055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:26.102016 2055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:26.105599 2055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:26.105818 2055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:26.111873 2055 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37841
I20260809 00:42:26.111959 2148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37841 every 8 connection(s)
I20260809 00:42:26.112345 2055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:26.113446 2149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:26.116580 2149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69: Bootstrap starting.
I20260809 00:42:26.117252 2149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:26.117568 2149 log.cc:824] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:26.118438 2149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69: No bootstrap required, opened a new log
I20260809 00:42:26.120296 2149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37841 } }
I20260809 00:42:26.120484 2149 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:26.120538 2149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e3d6353aa7a48dc92d9ff03d63fbc69, State: Initialized, Role: FOLLOWER
I20260809 00:42:26.120735 2149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37841 } }
I20260809 00:42:26.120841 2149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:26.120826 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 2055
I20260809 00:42:26.120886 2149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:26.120926 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:26.120941 2149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:26.121104 31139 external_mini_cluster.cc:1428] Setting key 78739bae05454f63280be4b5648ec460
I20260809 00:42:26.121814 2149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37841 } }
I20260809 00:42:26.121994 2149 leader_election.cc:302] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e3d6353aa7a48dc92d9ff03d63fbc69; no voters:
I20260809 00:42:26.122205 2149 leader_election.cc:288] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:26.122447 2158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:26.122488 2149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:26.122603 2158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [term 1 LEADER]: Becoming Leader. State: Replica: 6e3d6353aa7a48dc92d9ff03d63fbc69, State: Running, Role: LEADER
I20260809 00:42:26.122784 2158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37841 } }
I20260809 00:42:26.123490 2161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e3d6353aa7a48dc92d9ff03d63fbc69. Latest consensus state: current_term: 1 leader_uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37841 } } }
I20260809 00:42:26.123622 2161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:26.123661 2159 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e3d6353aa7a48dc92d9ff03d63fbc69" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37841 } } }
I20260809 00:42:26.123744 2159 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:26.123871 2181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:26.124570 2181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:26.126219 2181 catalog_manager.cc:1347] Generated new cluster ID: 07ec2569b93542ec926d8e1bb9ee21f5
I20260809 00:42:26.126281 2181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:26.133831 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:26.121722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33092 (local address 127.30.104.254:37841)
0809 00:42:26.121979 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:26.121990 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:26.122029 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:42:26.122167 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:26.122175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:26.122238 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:26.122394 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:26.123417 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.124921 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.126045 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.126589 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.126719 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:26.129630 (+ 2911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:26.129639 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:26.129639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:26.129668 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:26.132018 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:26.132491 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:26.132500 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:26.132500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:26.132632 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:26.132883 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:26.132884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:26.132884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:26.133166 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:26.133372 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:26.133445 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 00:42:26.133609 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":121,"threads_started":1}
I20260809 00:42:26.153602 2181 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:26.154215 2181 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:26.158943 2181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e3d6353aa7a48dc92d9ff03d63fbc69: Generated new TSK 0
I20260809 00:42:26.159168 2181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:26.198974 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37841
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:26.324244 2189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:26.324468 2189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:26.324494 2189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:26.325732 2189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:26.325780 2189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:26.325809 2189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:26.327368 2189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:26.327669 2189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:26.328009 2189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:26.333433 2201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:26.333436 2197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:26.333509 2196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:26.333775 2189 server_base.cc:1034] running on GCE node
I20260809 00:42:26.334004 2189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:26.334227 2189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:26.335496 2189 hybrid_clock.cc:648] HybridClock initialized: now 1786236146335474 us; error 33 us; skew 500 ppm
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236146, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:26.338254 2189 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:26.338680 2189 webserver.cc:460] Webserver started at http://127.30.104.193:34977/ using document root <none> and password file <none>
I20260809 00:42:26.338943 2189 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:26.338994 2189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:26.339125 2189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:26.340305 2189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "4942a036217b47f79333e2aa8e0a0474"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "a9ae4508b073182b2bfc13aca78b493c"
server_key_iv: "cdfdb666821219e77dde9fc8d82db2b7"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.340682 2189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "4942a036217b47f79333e2aa8e0a0474"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "a9ae4508b073182b2bfc13aca78b493c"
server_key_iv: "cdfdb666821219e77dde9fc8d82db2b7"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.342124 2189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:26.342995 2211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.343242 2189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:26.343353 2189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "4942a036217b47f79333e2aa8e0a0474"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "a9ae4508b073182b2bfc13aca78b493c"
server_key_iv: "cdfdb666821219e77dde9fc8d82db2b7"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.343518 2189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:26.360569 2189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:26.363018 2189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:26.363189 2189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:26.363592 2189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:26.363969 2189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:26.364027 2189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.364071 2189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:26.364097 2189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.371039 2189 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:45147
I20260809 00:42:26.371119 2376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:45147 every 8 connection(s)
I20260809 00:42:26.371548 2189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:26.376655 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 2189
I20260809 00:42:26.376773 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:26.377009 31139 external_mini_cluster.cc:1428] Setting key 83846f229a59320101d639868da16316
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236146, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:26.383808 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:26.372583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:44891 (local address 127.30.104.254:37841)
0809 00:42:26.372678 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:26.372681 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:26.372907 (+ 226us) server_negotiation.cc:407] Connection header received
0809 00:42:26.373447 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:26.373448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:26.373461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:26.373518 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:26.374872 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.375629 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.376976 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.377303 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.377381 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:26.380156 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:26.380159 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:26.380159 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:26.380168 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:26.381963 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:26.382496 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:26.382497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:26.382498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:26.382569 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:26.382901 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:26.382902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:26.382903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:26.383090 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:26.383177 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:26.383570 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 00:42:26.383679 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:42:26.383906 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:26.372532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37841 (local address 127.30.104.193:44891)
0809 00:42:26.372751 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:26.372798 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:26.373341 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:26.373602 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:26.373606 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:26.374527 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:26.374756 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:26.374762 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.375734 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.375735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:26.376878 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:26.376882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.377420 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.377422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:26.377581 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:26.378198 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:26.378209 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:26.380047 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:26.382069 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:26.382077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:26.382079 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:26.382348 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:26.382669 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:26.382670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:26.382670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:26.382836 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:26.383181 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:26.383187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:26.383341 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 00:42:26.383542 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 00:42:26.383703 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":119,"threads_started":1}
I20260809 00:42:26.384438 2378 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37841
I20260809 00:42:26.384569 2378 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:26.384800 2378 heartbeater.cc:511] Master 127.30.104.254:37841 requested a full tablet report, sending...
I20260809 00:42:26.385506 2100 ts_manager.cc:194] Registered new tserver with Master: 4942a036217b47f79333e2aa8e0a0474 (127.30.104.193:45147)
I20260809 00:42:26.386502 2100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:44891
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:26.406225 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37841
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:26.498041 2385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:26.498277 2385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:26.498306 2385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:26.499570 2385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:26.499616 2385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:26.499645 2385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:26.501150 2385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:26.501381 2385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:26.501582 2385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:26.507225 2395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:26.507208 2391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:26.507349 2385 server_base.cc:1034] running on GCE node
W20260809 00:42:26.507212 2392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:26.507758 2385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:26.508073 2385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:26.509234 2385 hybrid_clock.cc:648] HybridClock initialized: now 1786236146509207 us; error 46 us; skew 500 ppm
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236146, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:26.512287 2385 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:26.512845 2385 webserver.cc:460] Webserver started at http://127.30.104.194:44587/ using document root <none> and password file <none>
I20260809 00:42:26.513151 2385 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:26.513228 2385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:26.513404 2385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:26.514652 2385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "a899844e5b694311be45a8632ab1cf7b"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "6487c325398ad937e1e81ada74e3c5de"
server_key_iv: "1907cc5c3a5025131ac945e13c0bbbb6"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.515076 2385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "a899844e5b694311be45a8632ab1cf7b"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "6487c325398ad937e1e81ada74e3c5de"
server_key_iv: "1907cc5c3a5025131ac945e13c0bbbb6"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.516707 2385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 00:42:26.517571 2410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.517774 2385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:26.517890 2385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "a899844e5b694311be45a8632ab1cf7b"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "6487c325398ad937e1e81ada74e3c5de"
server_key_iv: "1907cc5c3a5025131ac945e13c0bbbb6"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.517974 2385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:26.543632 2385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:26.546325 2385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:26.546525 2385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:26.546813 2385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:26.547180 2385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:26.547236 2385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.547283 2385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:26.547317 2385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.554186 2385 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:42465
I20260809 00:42:26.554286 2565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:42465 every 8 connection(s)
I20260809 00:42:26.554652 2385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236146, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:26.562898 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 2385
I20260809 00:42:26.563035 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:26.563257 31139 external_mini_cluster.cc:1428] Setting key 4eade90f13a0f31dcbc230f05ec9eff4
I20260809 00:42:26.565107 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:26.555534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:43735 (local address 127.30.104.254:37841)
0809 00:42:26.555608 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:26.555611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:26.555782 (+ 171us) server_negotiation.cc:407] Connection header received
0809 00:42:26.556396 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:26.556397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:26.556410 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:26.556470 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:26.557498 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.558006 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.559040 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.559237 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.559312 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:26.561831 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:26.561833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:26.561833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:26.561847 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:26.563413 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:26.563994 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:26.563995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:26.563995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:26.564061 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:26.564338 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:26.564339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:26.564340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:26.564495 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:26.564599 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:26.564871 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 00:42:26.564975 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:26.565150 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:26.555405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37841 (local address 127.30.104.194:43735)
0809 00:42:26.555644 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:26.555696 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:26.556271 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:26.556561 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:26.556565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:26.557171 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:26.557387 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:26.557392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.558103 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.558104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:26.558943 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:26.558945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.559331 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.559332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:26.559500 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:26.560103 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:26.560111 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:26.561641 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:26.563517 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:26.563527 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:26.563530 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:26.563912 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:26.564133 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:26.564134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:26.564134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:26.564266 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:26.564602 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:26.564607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:26.564739 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 00:42:26.564830 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 00:42:26.564968 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":128,"threads_started":1}
I20260809 00:42:26.565670 2566 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37841
I20260809 00:42:26.565789 2566 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:26.566043 2566 heartbeater.cc:511] Master 127.30.104.254:37841 requested a full tablet report, sending...
I20260809 00:42:26.566522 2100 ts_manager.cc:194] Registered new tserver with Master: a899844e5b694311be45a8632ab1cf7b (127.30.104.194:42465)
I20260809 00:42:26.566962 2100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:43735
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:26.594314 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37841
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:26.681613 2574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:26.681845 2574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:26.681871 2574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:26.683071 2574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:26.683120 2574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:26.683147 2574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:26.684706 2574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37841
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:26.684973 2574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:26.685223 2574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:26.690730 2586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:26.690733 2591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:26.690868 2574 server_base.cc:1034] running on GCE node
W20260809 00:42:26.690718 2587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:26.691176 2574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:26.691391 2574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:26.692601 2574 hybrid_clock.cc:648] HybridClock initialized: now 1786236146692589 us; error 30 us; skew 500 ppm
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236146, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:26.695206 2574 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:26.695683 2574 webserver.cc:460] Webserver started at http://127.30.104.195:36537/ using document root <none> and password file <none>
I20260809 00:42:26.695919 2574 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:26.695969 2574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:26.696132 2574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:26.697278 2574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "2601ac5bca3c41f59e9e9e7d23806028"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "4adf862ec99ee195fde157e2de583a3b"
server_key_iv: "96ff06ee8fb6d40c385d17fdcd551305"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.697646 2574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "2601ac5bca3c41f59e9e9e7d23806028"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "4adf862ec99ee195fde157e2de583a3b"
server_key_iv: "96ff06ee8fb6d40c385d17fdcd551305"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.699047 2574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:26.699877 2599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.700078 2574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:26.700184 2574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "2601ac5bca3c41f59e9e9e7d23806028"
format_stamp: "Formatted at 2026-08-09 00:42:26 on dist-test-slave-c12x"
server_key: "4adf862ec99ee195fde157e2de583a3b"
server_key_iv: "96ff06ee8fb6d40c385d17fdcd551305"
server_key_version: "encryptionkey@0"
I20260809 00:42:26.700260 2574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:26.715263 2574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:26.717692 2574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:26.717859 2574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:26.718108 2574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:26.718434 2574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:26.718489 2574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.718531 2574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:26.718562 2574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:26.725129 2574 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:44133
I20260809 00:42:26.725220 2769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:44133 every 8 connection(s)
I20260809 00:42:26.725550 2574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:26.730638 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 2574
I20260809 00:42:26.730747 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:26.730947 31139 external_mini_cluster.cc:1428] Setting key 60f5ac04e3b4cbbfd7cb7dc8f4721011
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2034](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236146, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:26.735330 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:26.726349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:43329 (local address 127.30.104.254:37841)
0809 00:42:26.726423 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:26.726425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:26.726626 (+ 201us) server_negotiation.cc:407] Connection header received
0809 00:42:26.727114 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:26.727115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:26.727126 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:26.727192 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:26.728176 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.728673 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.729708 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.729925 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.730024 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:26.732636 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:26.732637 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:26.732638 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:26.732645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:26.733931 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:26.734356 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:26.734357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:26.734358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:26.734421 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:26.734674 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:26.734675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:26.734675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:26.734795 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:26.734868 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:26.735151 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 00:42:26.735235 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:26.735585 2773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:26.726319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37841 (local address 127.30.104.195:43329)
0809 00:42:26.726473 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:26.726516 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:26.727002 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:26.727248 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:26.727251 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:26.727867 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:26.728083 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:26.728089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.728768 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.728769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:26.729619 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:26.729621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:26.730051 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:26.730053 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:26.730198 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:26.730761 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:26.730772 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:26.732540 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:26.734029 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:26.734035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:26.734038 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:26.734279 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:26.734488 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:26.734488 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:26.734489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:26.734603 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:26.734858 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:26.734864 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:26.735013 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 00:42:26.735129 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 00:42:26.735298 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260809 00:42:26.736093 2770 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37841
I20260809 00:42:26.736193 2770 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:26.736407 2770 heartbeater.cc:511] Master 127.30.104.254:37841 requested a full tablet report, sending...
I20260809 00:42:26.736797 2100 ts_manager.cc:194] Registered new tserver with Master: 2601ac5bca3c41f59e9e9e7d23806028 (127.30.104.195:44133)
I20260809 00:42:26.737169 2100 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:43329
I20260809 00:42:26.743136 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:42:26.756472 2157 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:26.755305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58114 (local address 127.30.104.254:37841)
0809 00:42:26.755403 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:26.755405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:26.755488 (+ 83us) server_negotiation.cc:407] Connection header received
0809 00:42:26.755589 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:26.755590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:26.755600 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:26.755654 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:26.756379 (+ 725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58114: BlockingRecv error: recv got EOF from 127.0.0.1:58114 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 00:42:26.757298 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 2189
I20260809 00:42:26.769152 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 2385
I20260809 00:42:26.795673 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 2574
I20260809 00:42:26.805145 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 2055
2026-08-09T00:42:26Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2196 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2787](info): set up 2 sockets
Aug 09 00:42:26 dist-test-slave-c12x krb5kdc[2787](info): commencing operation
krb5kdc: starting...
W20260809 00:42:28.031265 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:28 dist-test-slave-c12x krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:28.163365 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44121
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:44121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:28.256661 2807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:28.256886 2807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:28.256908 2807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:28.258033 2807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:28.258071 2807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:28.258085 2807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:28.258098 2807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:28.258111 2807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:28.259721 2807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:44121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:28.259933 2807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:28.260118 2807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:28.265149 2816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:28.265136 2815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:28.265172 2818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:28.265184 2807 server_base.cc:1034] running on GCE node
I20260809 00:42:28.265691 2807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:28.266003 2807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:28.267180 2807 hybrid_clock.cc:648] HybridClock initialized: now 1786236148267172 us; error 36 us; skew 500 ppm
Aug 09 00:42:28 dist-test-slave-c12x krb5kdc[2787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236148, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:28.275216 2807 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:28.275758 2807 webserver.cc:460] Webserver started at http://127.30.104.254:45561/ using document root <none> and password file <none>
I20260809 00:42:28.275964 2807 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:28.276006 2807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:28.276110 2807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:28.277201 2807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "73defde14b6e45538f2ccf0e727356c7"
format_stamp: "Formatted at 2026-08-09 00:42:28 on dist-test-slave-c12x"
server_key: "f5a80fc33efb67d9e60c98adc037e376"
server_key_iv: "ac08c274f212a1e715b4c826af7083b2"
server_key_version: "encryptionkey@0"
I20260809 00:42:28.277529 2807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "73defde14b6e45538f2ccf0e727356c7"
format_stamp: "Formatted at 2026-08-09 00:42:28 on dist-test-slave-c12x"
server_key: "f5a80fc33efb67d9e60c98adc037e376"
server_key_iv: "ac08c274f212a1e715b4c826af7083b2"
server_key_version: "encryptionkey@0"
I20260809 00:42:28.278967 2807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:28.279932 2825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:28.280112 2807 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:28.280208 2807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "73defde14b6e45538f2ccf0e727356c7"
format_stamp: "Formatted at 2026-08-09 00:42:28 on dist-test-slave-c12x"
server_key: "f5a80fc33efb67d9e60c98adc037e376"
server_key_iv: "ac08c274f212a1e715b4c826af7083b2"
server_key_version: "encryptionkey@0"
I20260809 00:42:28.280270 2807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:28.306931 2807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:28.309295 2807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:28.309423 2807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:28.313956 2807 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:44121
I20260809 00:42:28.314028 2903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:44121 every 8 connection(s)
I20260809 00:42:28.314528 2807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:28.315269 2904 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:28.317754 2904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7: Bootstrap starting.
I20260809 00:42:28.318388 2904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:28.318696 2904 log.cc:824] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:28.319512 2904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7: No bootstrap required, opened a new log
I20260809 00:42:28.319979 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 2807
I20260809 00:42:28.320063 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:28.320345 31139 external_mini_cluster.cc:1428] Setting key df8225e914d14df3cc26b287ea1dc95c
I20260809 00:42:28.322101 2904 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73defde14b6e45538f2ccf0e727356c7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44121 } }
I20260809 00:42:28.322299 2904 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:28.322340 2904 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73defde14b6e45538f2ccf0e727356c7, State: Initialized, Role: FOLLOWER
I20260809 00:42:28.322490 2904 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73defde14b6e45538f2ccf0e727356c7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44121 } }
I20260809 00:42:28.322589 2904 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:28.322628 2904 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:28.322675 2904 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:28.323360 2904 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73defde14b6e45538f2ccf0e727356c7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44121 } }
I20260809 00:42:28.323619 2904 leader_election.cc:302] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73defde14b6e45538f2ccf0e727356c7; no voters:
I20260809 00:42:28.323817 2904 leader_election.cc:288] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:28.323848 2911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:28.323985 2911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [term 1 LEADER]: Becoming Leader. State: Replica: 73defde14b6e45538f2ccf0e727356c7, State: Running, Role: LEADER
I20260809 00:42:28.324105 2904 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:28.324267 2911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73defde14b6e45538f2ccf0e727356c7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44121 } }
I20260809 00:42:28.324707 2912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73defde14b6e45538f2ccf0e727356c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73defde14b6e45538f2ccf0e727356c7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44121 } } }
I20260809 00:42:28.324788 2912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:28.324985 2913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73defde14b6e45538f2ccf0e727356c7. Latest consensus state: current_term: 1 leader_uuid: "73defde14b6e45538f2ccf0e727356c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73defde14b6e45538f2ccf0e727356c7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44121 } } }
I20260809 00:42:28.325055 2913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:28.325428 2923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:28.326021 2923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:28.328042 2923 catalog_manager.cc:1347] Generated new cluster ID: e908c8e764fb4a4bad4161f89a56d305
I20260809 00:42:28.328107 2923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:42:28 dist-test-slave-c12x krb5kdc[2787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:28.334414 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:28.320887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.30.104.254:44121)
0809 00:42:28.321073 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:28.321081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:28.321107 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:42:28.322019 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:28.322027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:28.322079 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:28.322202 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:28.323102 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:28.324574 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:28.325764 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:28.326481 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:28.326607 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:28.329749 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:28.329753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:28.329754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:28.329766 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:28.332370 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:28.332784 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:28.332788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:28.332788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:28.332905 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:28.333079 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:28.333080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:28.333080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:28.333377 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:28.333519 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:28.334038 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 00:42:28.334186 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260809 00:42:28.335273 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:42:28.348904 2923 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:28.349622 2923 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:28.354600 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:28.349878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.30.104.254:44121)
0809 00:42:28.349922 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:28.349924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:28.350099 (+ 175us) server_negotiation.cc:407] Connection header received
0809 00:42:28.350167 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:28.350168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:28.350180 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:28.350239 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:28.350702 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:28.351492 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:28.352656 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:28.352967 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:28.353025 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:28.353945 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:28.353946 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:28.353947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:28.354181 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:28.354200 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:28.354208 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:28.354251 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:28.354480 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 00:42:28.354514 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:42:28.359660 2923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73defde14b6e45538f2ccf0e727356c7: Generated new TSK 0
I20260809 00:42:28.359901 2923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:42:28.366796 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 2807
2026-08-09T00:42:28Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1558 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:28 dist-test-slave-c12x krb5kdc[2955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:28 dist-test-slave-c12x krb5kdc[2955](info): set up 2 sockets
Aug 09 00:42:28 dist-test-slave-c12x krb5kdc[2955](info): commencing operation
krb5kdc: starting...
W20260809 00:42:29.592113 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:29 dist-test-slave-c12x krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:29.725858 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44683
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:44683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:29.812183 2972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:29.812454 2972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:29.812496 2972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:29.813709 2972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:29.813786 2972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:29.813812 2972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:29.813841 2972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:29.813896 2972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:29.815547 2972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:44683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:29.815858 2972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:29.816105 2972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:29.821405 2983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:29.821580 2972 server_base.cc:1034] running on GCE node
W20260809 00:42:29.821448 2981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:29.821439 2980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:29.821969 2972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:29.822240 2972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:29.823377 2972 hybrid_clock.cc:648] HybridClock initialized: now 1786236149823356 us; error 39 us; skew 500 ppm
Aug 09 00:42:29 dist-test-slave-c12x krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236149, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:29.826067 2972 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:29.826557 2972 webserver.cc:460] Webserver started at http://127.30.104.254:44337/ using document root <none> and password file <none>
I20260809 00:42:29.826808 2972 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:29.826856 2972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:29.827025 2972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:29.828207 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "d7f2364bb8e24ee5889b524feab4edee"
format_stamp: "Formatted at 2026-08-09 00:42:29 on dist-test-slave-c12x"
server_key: "8c87b6e86090e6365b32056b1810ec3c"
server_key_iv: "855d2c21e658ea9fd12b2a47b45deba7"
server_key_version: "encryptionkey@0"
I20260809 00:42:29.828590 2972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "d7f2364bb8e24ee5889b524feab4edee"
format_stamp: "Formatted at 2026-08-09 00:42:29 on dist-test-slave-c12x"
server_key: "8c87b6e86090e6365b32056b1810ec3c"
server_key_iv: "855d2c21e658ea9fd12b2a47b45deba7"
server_key_version: "encryptionkey@0"
I20260809 00:42:29.830045 2972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:29.830978 2993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:29.831365 2972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:29.831502 2972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "d7f2364bb8e24ee5889b524feab4edee"
format_stamp: "Formatted at 2026-08-09 00:42:29 on dist-test-slave-c12x"
server_key: "8c87b6e86090e6365b32056b1810ec3c"
server_key_iv: "855d2c21e658ea9fd12b2a47b45deba7"
server_key_version: "encryptionkey@0"
I20260809 00:42:29.831599 2972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:29.843626 2972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:29.845902 2972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:29.846095 2972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:29.850435 2972 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:44683
I20260809 00:42:29.850469 3073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:44683 every 8 connection(s)
I20260809 00:42:29.850859 2972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:29.851977 3074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:29.852051 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 2972
I20260809 00:42:29.852185 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:29.852389 31139 external_mini_cluster.cc:1428] Setting key a6ad9cc24abacc1c71182f41323ac616
I20260809 00:42:29.855540 3074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee: Bootstrap starting.
I20260809 00:42:29.856246 3074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:29.856571 3074 log.cc:824] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:29.857311 3074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee: No bootstrap required, opened a new log
Aug 09 00:42:29 dist-test-slave-c12x krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:29.859939 3074 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f2364bb8e24ee5889b524feab4edee" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44683 } }
I20260809 00:42:29.860090 3074 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:29.860112 3074 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7f2364bb8e24ee5889b524feab4edee, State: Initialized, Role: FOLLOWER
I20260809 00:42:29.860270 3074 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f2364bb8e24ee5889b524feab4edee" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44683 } }
I20260809 00:42:29.860378 3074 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:29.860414 3074 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:29.860471 3074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:29.861222 3074 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f2364bb8e24ee5889b524feab4edee" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44683 } }
I20260809 00:42:29.861361 3074 leader_election.cc:302] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7f2364bb8e24ee5889b524feab4edee; no voters:
I20260809 00:42:29.861543 3074 leader_election.cc:288] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:29.861605 3079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:29.861763 3079 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [term 1 LEADER]: Becoming Leader. State: Replica: d7f2364bb8e24ee5889b524feab4edee, State: Running, Role: LEADER
I20260809 00:42:29.861816 3074 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:29.861931 3079 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f2364bb8e24ee5889b524feab4edee" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44683 } }
I20260809 00:42:29.862455 3080 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7f2364bb8e24ee5889b524feab4edee. Latest consensus state: current_term: 1 leader_uuid: "d7f2364bb8e24ee5889b524feab4edee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f2364bb8e24ee5889b524feab4edee" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44683 } } }
I20260809 00:42:29.862578 3080 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:29.862669 3082 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7f2364bb8e24ee5889b524feab4edee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7f2364bb8e24ee5889b524feab4edee" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44683 } } }
I20260809 00:42:29.862800 3082 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:29.863047 3096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:29.863693 3096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:29.865311 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:29.852965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.30.104.254:44683)
0809 00:42:29.853125 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:29.853133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:29.853159 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:42:29.853638 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:29.853645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:29.853772 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:29.853912 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:29.854777 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:29.855855 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:29.856982 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:29.857469 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:29.857556 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:29.860317 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:29.860321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:29.860321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:29.860334 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:29.863332 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:29.863923 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:29.863926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:29.863927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:29.864033 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:29.864273 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:29.864274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:29.864274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:29.864597 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:29.864735 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:29.865023 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 00:42:29.865171 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":76,"threads_started":1}
I20260809 00:42:29.865540 3096 catalog_manager.cc:1347] Generated new cluster ID: 235a7907abd748af9712dd7b97c1ecda
I20260809 00:42:29.865605 3096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:29.888729 3096 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:29.889370 3096 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:29.893996 3096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7f2364bb8e24ee5889b524feab4edee: Generated new TSK 0
I20260809 00:42:29.894183 3096 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:29.926707 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44683
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:30.012959 3113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:30.013163 3113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:30.013187 3113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:30.014348 3113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:30.014393 3113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:30.014420 3113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:30.015982 3113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:30.016229 3113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:30.016484 3113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:30.021790 3121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:30.021858 3113 server_base.cc:1034] running on GCE node
W20260809 00:42:30.021765 3125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:30.021793 3122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:30.022193 3113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:30.022455 3113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:30.023617 3113 hybrid_clock.cc:648] HybridClock initialized: now 1786236150023613 us; error 43 us; skew 500 ppm
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:30.026288 3113 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:30.026804 3113 webserver.cc:460] Webserver started at http://127.30.104.193:42247/ using document root <none> and password file <none>
I20260809 00:42:30.027077 3113 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:30.027150 3113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:30.027297 3113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:30.028471 3113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "84095587ebb447619ba5f4346924b1c8"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "674bf3c2676e00fe492bb189b048b542"
server_key_iv: "67b61e5ddd78429a785c7d91279dcd4d"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.028855 3113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "84095587ebb447619ba5f4346924b1c8"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "674bf3c2676e00fe492bb189b048b542"
server_key_iv: "67b61e5ddd78429a785c7d91279dcd4d"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.030344 3113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:30.031152 3135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.031332 3113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:30.031421 3113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "84095587ebb447619ba5f4346924b1c8"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "674bf3c2676e00fe492bb189b048b542"
server_key_iv: "67b61e5ddd78429a785c7d91279dcd4d"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.031540 3113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:30.055222 3113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:30.057648 3113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:30.057819 3113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:30.058081 3113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:30.058410 3113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:30.058463 3113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.058506 3113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:30.058537 3113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.065025 3113 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:36423
I20260809 00:42:30.065081 3292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:36423 every 8 connection(s)
I20260809 00:42:30.065435 3113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:30.073601 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 3113
I20260809 00:42:30.073727 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:30.073964 31139 external_mini_cluster.cc:1428] Setting key 4d61d9e84d442ad463019ba39a629f68
I20260809 00:42:30.075819 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:30.066203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:43453 (local address 127.30.104.254:44683)
0809 00:42:30.066247 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:30.066249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:30.066481 (+ 232us) server_negotiation.cc:407] Connection header received
0809 00:42:30.067056 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:30.067058 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:30.067072 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:30.067125 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:30.068181 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.068704 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.069755 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.069986 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.070063 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:30.072333 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:30.072334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:30.072335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:30.072343 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:30.074080 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:30.074591 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:30.074593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:30.074593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:30.074662 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:30.074994 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:30.074995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:30.074995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:30.075114 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:30.075172 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:30.075582 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 00:42:30.075744 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:42:30.076175 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:30.066177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44683 (local address 127.30.104.193:43453)
0809 00:42:30.066343 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:30.066387 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:30.066936 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:30.067198 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:30.067202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:30.067862 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:30.068072 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:30.068077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.068807 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.068808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:30.069662 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:30.069664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.070071 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.070073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:30.070198 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:30.070726 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:30.070735 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:30.072227 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:30.074204 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:30.074213 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:30.074216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:30.074501 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:30.074747 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:30.074747 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:30.074748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:30.074918 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:30.075183 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:30.075191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:30.075371 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 00:42:30.075557 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 00:42:30.075872 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260809 00:42:30.076771 3293 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44683
I20260809 00:42:30.076898 3293 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:30.077098 3293 heartbeater.cc:511] Master 127.30.104.254:44683 requested a full tablet report, sending...
I20260809 00:42:30.077682 3018 ts_manager.cc:194] Registered new tserver with Master: 84095587ebb447619ba5f4346924b1c8 (127.30.104.193:36423)
I20260809 00:42:30.078331 3018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:43453
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:30.102883 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44683
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:30.187849 3300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:30.188072 3300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:30.188098 3300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:30.189313 3300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:30.189363 3300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:30.189392 3300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:30.190879 3300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:30.191129 3300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:30.191372 3300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:30.196853 3305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:30.196779 3309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:30.196961 3300 server_base.cc:1034] running on GCE node
W20260809 00:42:30.196766 3306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:30.197320 3300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:30.197613 3300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:30.198809 3300 hybrid_clock.cc:648] HybridClock initialized: now 1786236150198781 us; error 41 us; skew 500 ppm
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:30.201920 3300 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:30.202343 3300 webserver.cc:460] Webserver started at http://127.30.104.194:33791/ using document root <none> and password file <none>
I20260809 00:42:30.202566 3300 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:30.202610 3300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:30.202703 3300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:30.203836 3300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "08c244b035fa4d198b713260a24e8e5a"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "8a34df35fd26869025b1333eb712e7f7"
server_key_iv: "905341b9551ed6250d3df8d458662caf"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.204157 3300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "08c244b035fa4d198b713260a24e8e5a"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "8a34df35fd26869025b1333eb712e7f7"
server_key_iv: "905341b9551ed6250d3df8d458662caf"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.205571 3300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:30.206250 3317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.206446 3300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:30.206548 3300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "08c244b035fa4d198b713260a24e8e5a"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "8a34df35fd26869025b1333eb712e7f7"
server_key_iv: "905341b9551ed6250d3df8d458662caf"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.206607 3300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:30.222293 3300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:30.224583 3300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:30.224699 3300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:30.224892 3300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:30.225153 3300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:30.225185 3300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.225207 3300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:30.225219 3300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.231030 3300 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:35341
I20260809 00:42:30.231108 3472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:35341 every 8 connection(s)
I20260809 00:42:30.231449 3300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:30.239022 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 3300
I20260809 00:42:30.239152 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:30.239405 31139 external_mini_cluster.cc:1428] Setting key a01ef51fd70cacba0f9b19149d38cddd
I20260809 00:42:30.241560 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:30.232213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:39127 (local address 127.30.104.254:44683)
0809 00:42:30.232277 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:30.232279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:30.232498 (+ 219us) server_negotiation.cc:407] Connection header received
0809 00:42:30.232973 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:30.232974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:30.232983 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:30.233046 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:30.234023 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.234542 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.235871 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.236132 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.236197 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:30.238413 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:30.238414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:30.238415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:30.238428 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:30.240065 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:30.240533 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:30.240535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:30.240535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:30.240592 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:30.240853 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:30.240854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:30.240854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:30.240979 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:30.241083 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:30.241362 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 00:42:30.241459 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:42:30.241878 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:30.232183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44683 (local address 127.30.104.194:39127)
0809 00:42:30.232369 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:30.232406 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:30.232889 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:30.233100 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:30.233104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:30.233681 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:30.233941 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:30.233947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.234639 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.234641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:30.235751 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:30.235754 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.236215 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.236216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:30.236330 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:30.236848 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:30.236857 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:30.238301 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:30.240160 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:30.240168 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:30.240170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:30.240412 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:30.240663 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:30.240664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:30.240664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:30.240783 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:30.241072 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:30.241079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:30.241227 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 00:42:30.241338 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:42:30.241515 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260809 00:42:30.242496 3474 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44683
I20260809 00:42:30.242620 3474 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:30.242816 3474 heartbeater.cc:511] Master 127.30.104.254:44683 requested a full tablet report, sending...
I20260809 00:42:30.243245 3018 ts_manager.cc:194] Registered new tserver with Master: 08c244b035fa4d198b713260a24e8e5a (127.30.104.194:35341)
I20260809 00:42:30.243825 3018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:39127
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:30.267987 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44683
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:30.355868 3479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:30.356123 3479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:30.356151 3479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:30.357362 3479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:30.357409 3479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:30.357437 3479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:30.358974 3479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:30.359251 3479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:30.359527 3479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:30.365128 3490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:30.365082 3486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:30.365262 3479 server_base.cc:1034] running on GCE node
W20260809 00:42:30.365100 3487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:30.365533 3479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:30.365825 3479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:30.366992 3479 hybrid_clock.cc:648] HybridClock initialized: now 1786236150366987 us; error 37 us; skew 500 ppm
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:30.369748 3479 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:30.370235 3479 webserver.cc:460] Webserver started at http://127.30.104.195:43109/ using document root <none> and password file <none>
I20260809 00:42:30.370484 3479 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:30.370532 3479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:30.370675 3479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:30.371876 3479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "1f81bf5bbd8c4fe5bb1882dff918724f"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "2e164d901756d8380919eb809555a6cb"
server_key_iv: "0b54302d2242d75c93088a41d4ad29b4"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.372251 3479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "1f81bf5bbd8c4fe5bb1882dff918724f"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "2e164d901756d8380919eb809555a6cb"
server_key_iv: "0b54302d2242d75c93088a41d4ad29b4"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.373652 3479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:30.374466 3498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.374686 3479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:30.374783 3479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "1f81bf5bbd8c4fe5bb1882dff918724f"
format_stamp: "Formatted at 2026-08-09 00:42:30 on dist-test-slave-c12x"
server_key: "2e164d901756d8380919eb809555a6cb"
server_key_iv: "0b54302d2242d75c93088a41d4ad29b4"
server_key_version: "encryptionkey@0"
I20260809 00:42:30.374858 3479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:30.403625 3479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:30.406083 3479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:30.406252 3479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:30.406522 3479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:30.406874 3479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:30.406919 3479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.406942 3479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:30.406957 3479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:30.413326 3479 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:42419
I20260809 00:42:30.413368 3654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:42419 every 8 connection(s)
I20260809 00:42:30.413717 3479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:30.414613 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 3479
I20260809 00:42:30.414728 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:30.414934 31139 external_mini_cluster.cc:1428] Setting key 043c67ba3d7cf2122333c1aabf7f8ce1
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:30.423828 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:30.414479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:60739 (local address 127.30.104.254:44683)
0809 00:42:30.414533 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:30.414535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:30.414729 (+ 194us) server_negotiation.cc:407] Connection header received
0809 00:42:30.415338 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:30.415339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:30.415352 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:30.415459 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:30.416462 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.416971 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.418073 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.418279 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.418348 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:30.420705 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:30.420706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:30.420706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:30.420714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:30.422207 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:30.422660 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:30.422661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:30.422661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:30.422716 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:30.422969 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:30.422969 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:30.422970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:30.423094 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:30.423153 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:30.423671 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 00:42:30.423755 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:42:30.424047 3657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:30.414415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44683 (local address 127.30.104.195:60739)
0809 00:42:30.414585 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:30.414629 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:30.415223 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:30.415538 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:30.415543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:30.416147 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:30.416351 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:30.416357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.417073 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.417074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:30.417950 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:30.417954 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.418365 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.418365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:30.418486 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:30.419022 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:30.419031 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:30.420589 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:30.422303 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:30.422311 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:30.422314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:30.422584 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:30.422780 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:30.422781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:30.422781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:30.422898 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:30.423181 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:30.423187 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:30.423341 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 00:42:30.423626 (+ 285us) client_negotiation.cc:240] Negotiation successful
0809 00:42:30.423780 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260809 00:42:30.424641 3655 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44683
I20260809 00:42:30.424777 3655 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:30.425007 3655 heartbeater.cc:511] Master 127.30.104.254:44683 requested a full tablet report, sending...
I20260809 00:42:30.425397 3018 ts_manager.cc:194] Registered new tserver with Master: 1f81bf5bbd8c4fe5bb1882dff918724f (127.30.104.195:42419)
I20260809 00:42:30.425792 3018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:60739
I20260809 00:42:30.426678 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[2955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236150, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[2955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236150, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:30.452639 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:30.444966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52322 (local address 127.30.104.254:44683)
0809 00:42:30.445043 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:30.445046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:30.445148 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:42:30.445186 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:30.445187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:30.445198 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:30.445263 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:30.445877 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.446434 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.447217 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:30.447516 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:30.447584 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:30.449934 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:30.449935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:30.449935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:30.449943 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:30.451511 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:30.451868 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:30.451869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:30.451869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:30.451931 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:30.452096 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:30.452097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:30.452097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:30.452250 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:30.452328 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:30.452441 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:42:30.452508 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 00:42:30.452989 3018 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52322
I20260809 00:42:30.453595 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 3113
I20260809 00:42:30.472749 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 3300
I20260809 00:42:30.504099 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 3479
I20260809 00:42:30.516996 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 2972
2026-08-09T00:42:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2152 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[3674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[3674](info): set up 2 sockets
Aug 09 00:42:30 dist-test-slave-c12x krb5kdc[3674](info): commencing operation
krb5kdc: starting...
W20260809 00:42:31.762543 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:31 dist-test-slave-c12x krb5kdc[3674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:31.892608 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:36083
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:44145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:36083
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:31.979633 3691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:31.979914 3691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:31.979967 3691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:31.981213 3691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:31.981297 3691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:31.981320 3691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:31.981343 3691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:31.981370 3691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:31.983036 3691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:44145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:36083
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:36083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:31.983349 3691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:31.983633 3691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:31.988864 3697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:31.988865 3701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:31.989040 3698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:31.988955 3691 server_base.cc:1034] running on GCE node
I20260809 00:42:31.989475 3691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:31.989799 3691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:31.990948 3691 hybrid_clock.cc:648] HybridClock initialized: now 1786236151990931 us; error 36 us; skew 500 ppm
Aug 09 00:42:31 dist-test-slave-c12x krb5kdc[3674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236151, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:31.993754 3691 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:31.994240 3691 webserver.cc:460] Webserver started at http://127.30.104.254:37959/ using document root <none> and password file <none>
I20260809 00:42:31.994508 3691 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:31.994557 3691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:31.994724 3691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:31.995941 3691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "c467db8f107a4e7cab586f357f938eb3"
format_stamp: "Formatted at 2026-08-09 00:42:31 on dist-test-slave-c12x"
server_key: "062f6fdeac14ca5e7f885b90f21c767c"
server_key_iv: "f877fe45145e2d3cbc4c57a06307877d"
server_key_version: "encryptionkey@0"
I20260809 00:42:31.996313 3691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "c467db8f107a4e7cab586f357f938eb3"
format_stamp: "Formatted at 2026-08-09 00:42:31 on dist-test-slave-c12x"
server_key: "062f6fdeac14ca5e7f885b90f21c767c"
server_key_iv: "f877fe45145e2d3cbc4c57a06307877d"
server_key_version: "encryptionkey@0"
I20260809 00:42:31.997854 3691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:31.998729 3713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:31.998965 3691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:31.999068 3691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "c467db8f107a4e7cab586f357f938eb3"
format_stamp: "Formatted at 2026-08-09 00:42:31 on dist-test-slave-c12x"
server_key: "062f6fdeac14ca5e7f885b90f21c767c"
server_key_iv: "f877fe45145e2d3cbc4c57a06307877d"
server_key_version: "encryptionkey@0"
I20260809 00:42:31.999142 3691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:32.017819 3691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:32.020471 3691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:32.020603 3691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:32.024781 3691 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:36083
I20260809 00:42:32.025184 3691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:32.025197 3783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:36083 every 8 connection(s)
I20260809 00:42:32.025954 3784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:32.028420 3784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3: Bootstrap starting.
I20260809 00:42:32.029034 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 3691
I20260809 00:42:32.029065 3784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:32.029145 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:32.029415 31139 external_mini_cluster.cc:1428] Setting key 2c0545f4863ee07455a271bad8365c56
I20260809 00:42:32.029417 3784 log.cc:824] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:32.030333 3784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3: No bootstrap required, opened a new log
I20260809 00:42:32.033265 3784 raft_consensus.cc:348] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c467db8f107a4e7cab586f357f938eb3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36083 } }
I20260809 00:42:32.033490 3784 raft_consensus.cc:374] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:32.033540 3784 raft_consensus.cc:729] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c467db8f107a4e7cab586f357f938eb3, State: Initialized, Role: FOLLOWER
I20260809 00:42:32.033691 3784 consensus_queue.cc:260] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c467db8f107a4e7cab586f357f938eb3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36083 } }
I20260809 00:42:32.033777 3784 raft_consensus.cc:388] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:32.033826 3784 raft_consensus.cc:482] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:32.033882 3784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:32.034646 3784 raft_consensus.cc:504] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c467db8f107a4e7cab586f357f938eb3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36083 } }
I20260809 00:42:32.034773 3784 leader_election.cc:302] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c467db8f107a4e7cab586f357f938eb3; no voters:
I20260809 00:42:32.034966 3784 leader_election.cc:288] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:32.035077 3791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:32.035266 3784 sys_catalog.cc:564] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:32.035283 3791 raft_consensus.cc:686] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [term 1 LEADER]: Becoming Leader. State: Replica: c467db8f107a4e7cab586f357f938eb3, State: Running, Role: LEADER
I20260809 00:42:32.035552 3791 consensus_queue.cc:237] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c467db8f107a4e7cab586f357f938eb3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36083 } }
I20260809 00:42:32.035983 3792 sys_catalog.cc:455] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c467db8f107a4e7cab586f357f938eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c467db8f107a4e7cab586f357f938eb3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36083 } } }
I20260809 00:42:32.036150 3792 sys_catalog.cc:458] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:32.036383 3794 sys_catalog.cc:455] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c467db8f107a4e7cab586f357f938eb3. Latest consensus state: current_term: 1 leader_uuid: "c467db8f107a4e7cab586f357f938eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c467db8f107a4e7cab586f357f938eb3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36083 } } }
I20260809 00:42:32.036525 3794 sys_catalog.cc:458] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:32.036695 3797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:32.037251 3797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:32.039089 3797 catalog_manager.cc:1347] Generated new cluster ID: dafe886d434d48ec8a7841d33ba37bef
I20260809 00:42:32.039147 3797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:32.042301 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.029956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.30.104.254:36083)
0809 00:42:32.030198 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:32.030208 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:32.030245 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:42:32.030863 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:32.030873 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:32.030948 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:32.031137 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:32.031869 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.033357 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.034358 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.035044 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.035611 (+ 567us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.038532 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:32.038535 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:32.038535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:32.038567 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:32.040587 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.041021 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.041024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.041024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.041124 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.041384 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.041385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.041385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.041655 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:32.041854 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:32.041960 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 00:42:32.042116 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":120,"threads_started":1}
I20260809 00:42:32.066998 3797 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:32.067696 3797 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:32.076287 3797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c467db8f107a4e7cab586f357f938eb3: Generated new TSK 0
I20260809 00:42:32.076534 3797 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:32.118806 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36083
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:44145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:32.215569 3821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:32.215801 3821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:32.215827 3821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:32.217015 3821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:32.217056 3821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:32.217083 3821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:32.218604 3821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:44145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:32.218824 3821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:32.219025 3821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:32.224345 3829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:32.224327 3833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:32.224453 3821 server_base.cc:1034] running on GCE node
W20260809 00:42:32.224553 3830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:32.224874 3821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:32.225168 3821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:32.226400 3821 hybrid_clock.cc:648] HybridClock initialized: now 1786236152226387 us; error 44 us; skew 500 ppm
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:32.229250 3821 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:32.229786 3821 webserver.cc:460] Webserver started at http://127.30.104.193:35931/ using document root <none> and password file <none>
I20260809 00:42:32.230044 3821 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:32.230091 3821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:32.230192 3821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:32.231325 3821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "3b9832eda4c5437fbedf8ff73ca89a07"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "66822a472a86e0a4cd15ef0ba819f1da"
server_key_iv: "505ad4bfa7aeb9895e40885715e31a4a"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.231752 3821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "3b9832eda4c5437fbedf8ff73ca89a07"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "66822a472a86e0a4cd15ef0ba819f1da"
server_key_iv: "505ad4bfa7aeb9895e40885715e31a4a"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.233430 3821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:32.234220 3844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.234423 3821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:32.234541 3821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "3b9832eda4c5437fbedf8ff73ca89a07"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "66822a472a86e0a4cd15ef0ba819f1da"
server_key_iv: "505ad4bfa7aeb9895e40885715e31a4a"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.234622 3821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:32.256172 3821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:32.258706 3821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:32.258872 3821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:32.259147 3821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:32.259557 3821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:32.259598 3821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.259621 3821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:32.259662 3821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.265836 3821 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:33521
I20260809 00:42:32.265918 3996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:33521 every 8 connection(s)
I20260809 00:42:32.266259 3821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:32.276096 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 3821
I20260809 00:42:32.276206 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.267250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:52835 (local address 127.30.104.254:36083)
0809 00:42:32.267329 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:32.267332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:32.267542 (+ 210us) server_negotiation.cc:407] Connection header received
0809 00:42:32.268514 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:32.268515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:32.268527 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:32.268584 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:32.269211 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.269717 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.270697 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.270914 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.270975 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.273269 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:32.273271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:32.273271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:32.273278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:32.274691 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.275089 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.275091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.275091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.275156 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.275421 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.275422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.275422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.275595 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:32.275697 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:32.275989 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 00:42:32.276087 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:42:32.276217 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:32.276321 4002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.267199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36083 (local address 127.30.104.193:52835)
0809 00:42:32.267375 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:32.267451 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:32.268414 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:32.268653 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:32.268656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:32.268920 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:32.269134 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:32.269138 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.269793 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.269794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:32.270596 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:32.270598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.270993 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.270994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:32.271103 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.271747 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:32.271756 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:32.273179 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:32.274766 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:32.274772 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:32.274774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:32.274991 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:32.275226 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:32.275226 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:32.275227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:32.275328 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:32.275683 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:32.275688 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:32.275834 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 00:42:32.275938 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 00:42:32.276081 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260809 00:42:32.276435 31139 external_mini_cluster.cc:1428] Setting key 4ca8006d00acca8ee73fc5218233dbf0
I20260809 00:42:32.277120 3997 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36083
I20260809 00:42:32.277240 3997 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:32.277458 3997 heartbeater.cc:511] Master 127.30.104.254:36083 requested a full tablet report, sending...
I20260809 00:42:32.278064 3734 ts_manager.cc:194] Registered new tserver with Master: 3b9832eda4c5437fbedf8ff73ca89a07 (127.30.104.193:33521)
I20260809 00:42:32.278759 3734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:52835
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:32.306102 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36083
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:44145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:32.393234 4007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:32.393527 4007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:32.393575 4007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:32.395048 4007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:32.395114 4007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:32.395170 4007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:32.398234 4007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:44145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:32.398588 4007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:32.398962 4007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:32.407069 4014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:32.407090 4017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:32.407516 4013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:32.407784 4007 server_base.cc:1034] running on GCE node
I20260809 00:42:32.408011 4007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:32.408327 4007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:32.409478 4007 hybrid_clock.cc:648] HybridClock initialized: now 1786236152409462 us; error 30 us; skew 500 ppm
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:32.412078 4007 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:32.412580 4007 webserver.cc:460] Webserver started at http://127.30.104.194:36655/ using document root <none> and password file <none>
I20260809 00:42:32.412842 4007 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:32.412911 4007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:32.413065 4007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:32.414170 4007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "b0209ade3c4c42aea5bdaff7f8429481"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "92fc737fe0dda168ea6a68a14834aed8"
server_key_iv: "e05362f912e2785056dd519b9d2b7993"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.414541 4007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "b0209ade3c4c42aea5bdaff7f8429481"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "92fc737fe0dda168ea6a68a14834aed8"
server_key_iv: "e05362f912e2785056dd519b9d2b7993"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.415988 4007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:32.416698 4026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.416868 4007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:42:32.416958 4007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "b0209ade3c4c42aea5bdaff7f8429481"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "92fc737fe0dda168ea6a68a14834aed8"
server_key_iv: "e05362f912e2785056dd519b9d2b7993"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.417037 4007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:32.425143 4007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:32.427224 4007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:32.427402 4007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:32.427734 4007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:32.428068 4007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:32.428121 4007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.428162 4007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:32.428191 4007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.434634 4007 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:38623
I20260809 00:42:32.434675 4190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:38623 every 8 connection(s)
I20260809 00:42:32.435057 4007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:32.444916 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.435931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:32989 (local address 127.30.104.254:36083)
0809 00:42:32.436003 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:32.436007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:32.436210 (+ 203us) server_negotiation.cc:407] Connection header received
0809 00:42:32.437307 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:32.437308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:32.437320 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:32.437374 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:32.438027 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.438526 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.439615 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.439802 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.439852 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.441972 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:32.441974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:32.441974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:32.441983 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:32.443598 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.444000 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.444001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.444001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.444064 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.444297 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.444297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.444298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.444419 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:32.444478 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:32.444760 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 00:42:32.444847 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:42:32.445019 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.435905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36083 (local address 127.30.104.194:32989)
0809 00:42:32.436084 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:32.436120 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:32.437202 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:32.437444 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:32.437447 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:32.437710 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:32.437959 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:32.437964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.438608 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.438609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:32.439537 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:32.439540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.439868 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.439869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:32.439971 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.440475 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:32.440483 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:32.441862 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:32.443684 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:32.443693 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:32.443695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:32.443912 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:32.444141 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:32.444142 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:32.444142 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:32.444242 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:32.444481 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:32.444486 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:32.444603 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 00:42:32.444692 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:42:32.444815 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":113,"threads_started":1}
I20260809 00:42:32.445184 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 4007
I20260809 00:42:32.445295 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:32.445523 31139 external_mini_cluster.cc:1428] Setting key b8d65955caf78b42c040428b621e84f2
I20260809 00:42:32.445539 4193 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36083
I20260809 00:42:32.445643 4193 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:32.445819 4193 heartbeater.cc:511] Master 127.30.104.254:36083 requested a full tablet report, sending...
I20260809 00:42:32.446239 3734 ts_manager.cc:194] Registered new tserver with Master: b0209ade3c4c42aea5bdaff7f8429481 (127.30.104.194:38623)
I20260809 00:42:32.446653 3734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:32989
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:32.474889 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36083
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:44145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:32.560748 4200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:32.560963 4200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:32.560987 4200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:32.562112 4200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:32.562153 4200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:32.562180 4200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:32.563760 4200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:44145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:32.563958 4200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:32.564155 4200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:32.569406 4207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:32.569406 4210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:32.569545 4208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:32.569646 4200 server_base.cc:1034] running on GCE node
I20260809 00:42:32.569904 4200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:32.570140 4200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:32.571343 4200 hybrid_clock.cc:648] HybridClock initialized: now 1786236152571332 us; error 34 us; skew 500 ppm
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:32.574137 4200 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:32.574558 4200 webserver.cc:460] Webserver started at http://127.30.104.195:39291/ using document root <none> and password file <none>
I20260809 00:42:32.574772 4200 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:32.574815 4200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:32.574925 4200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:32.576071 4200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "16d9463c3e354c01a14c1dfae8bb878c"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "46a53948ceb7a951b6e6cc988558f5ff"
server_key_iv: "5bd4bf78334153274ae2c8baced11e14"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.576491 4200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "16d9463c3e354c01a14c1dfae8bb878c"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "46a53948ceb7a951b6e6cc988558f5ff"
server_key_iv: "5bd4bf78334153274ae2c8baced11e14"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.577883 4200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:32.578675 4221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.578860 4200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:32.578935 4200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "16d9463c3e354c01a14c1dfae8bb878c"
format_stamp: "Formatted at 2026-08-09 00:42:32 on dist-test-slave-c12x"
server_key: "46a53948ceb7a951b6e6cc988558f5ff"
server_key_iv: "5bd4bf78334153274ae2c8baced11e14"
server_key_version: "encryptionkey@0"
I20260809 00:42:32.578990 4200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:32.595407 4200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:32.597460 4200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:32.597584 4200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:32.597785 4200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:32.598115 4200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:32.598150 4200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.598192 4200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:32.598222 4200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:32.604475 4200 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:44825
I20260809 00:42:32.604555 4381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:44825 every 8 connection(s)
I20260809 00:42:32.604898 4200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:32.611244 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 4200
I20260809 00:42:32.611334 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:32.611567 31139 external_mini_cluster.cc:1428] Setting key 6c8f1362e49d837b9ccce6b2af72dfd5
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:32.614990 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.605894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:49495 (local address 127.30.104.254:36083)
0809 00:42:32.605977 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:32.605980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:32.606081 (+ 101us) server_negotiation.cc:407] Connection header received
0809 00:42:32.607115 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:32.607116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:32.607128 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:32.607197 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:32.607935 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.608444 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.609466 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.609653 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.609705 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.612057 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:32.612058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:32.612059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:32.612067 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:32.613578 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.614012 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.614014 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.614014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.614077 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.614306 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.614307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.614307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.614421 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:32.614503 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:32.614841 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 00:42:32.614917 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 00:42:32.615149 4384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.605753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36083 (local address 127.30.104.195:49495)
0809 00:42:32.605941 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:32.605991 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:32.606982 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:32.607273 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:32.607277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:32.607630 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:32.607841 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:32.607846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.608533 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.608533 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:32.609391 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:32.609393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.609729 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.609730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:32.609839 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.610433 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:32.610442 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:32.611972 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:32.613653 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:32.613660 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:32.613662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:32.613902 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:32.614146 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:32.614147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:32.614147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:32.614249 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:32.614500 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:32.614505 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:32.614676 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 00:42:32.614806 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 00:42:32.614943 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260809 00:42:32.615684 4382 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36083
I20260809 00:42:32.615789 4382 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:32.615989 4382 heartbeater.cc:511] Master 127.30.104.254:36083 requested a full tablet report, sending...
I20260809 00:42:32.616377 3734 ts_manager.cc:194] Registered new tserver with Master: 16d9463c3e354c01a14c1dfae8bb878c (127.30.104.195:44825)
I20260809 00:42:32.616766 3734 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:49495
I20260809 00:42:32.623476 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:32.649542 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.640925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33142 (local address 127.30.104.193:33521)
0809 00:42:32.641093 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:32.641100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:32.641119 (+ 19us) server_negotiation.cc:407] Connection header received
0809 00:42:32.641149 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:32.641151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:32.641165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:32.641232 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:32.641810 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.642547 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.643291 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.643700 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.643759 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.646182 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:32.646192 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:32.646192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:32.646200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:32.648115 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.648552 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.648556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.648556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.648626 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.648873 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.648874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.648874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.649037 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:32.649163 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:32.649301 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 00:42:32.649379 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[3674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236152, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:32.658442 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:32.650929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35020 (local address 127.30.104.254:36083)
0809 00:42:32.650991 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:32.650993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:32.651002 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:32.651028 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:32.651029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:32.651045 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:32.651093 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:32.651688 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.652192 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.652938 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:32.653141 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:32.653207 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:32.655652 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:32.655654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:32.655654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:32.655663 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:32.657312 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.657660 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.657661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.657661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.657716 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:32.657911 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:32.657911 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:32.657912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:32.658076 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:32.658160 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:32.658259 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:42:32.658324 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 00:42:32.658742 3734 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35020
I20260809 00:42:32.659161 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 3821
I20260809 00:42:32.670639 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 4007
I20260809 00:42:32.682081 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 4200
I20260809 00:42:32.690958 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 3691
2026-08-09T00:42:32Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2169 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[4399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[4399](info): set up 2 sockets
Aug 09 00:42:32 dist-test-slave-c12x krb5kdc[4399](info): commencing operation
krb5kdc: starting...
W20260809 00:42:33.921430 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:34.055698 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:36425
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:36425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:34.140028 4418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:34.140309 4418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:34.140352 4418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:34.141531 4418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:34.141601 4418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:34.141625 4418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:34.141649 4418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:34.141675 4418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:34.143226 4418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:36425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:36425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:34.143580 4418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:34.143826 4418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:34.148823 4427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:34.148969 4418 server_base.cc:1034] running on GCE node
W20260809 00:42:34.148836 4428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:34.148839 4430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:34.149341 4418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:34.149663 4418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:34.150827 4418 hybrid_clock.cc:648] HybridClock initialized: now 1786236154150812 us; error 42 us; skew 500 ppm
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:34.157316 4418 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:34.157703 4418 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:34.157757 4418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:34.157855 4418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:34.158918 4418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "6dbf80e234e34512a63c317fef2d9992"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "0d9d5bdb48363c612336622b978b1b38"
server_key_iv: "ca740ca2544448c0005e9f2f7f513166"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.159250 4418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "6dbf80e234e34512a63c317fef2d9992"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "0d9d5bdb48363c612336622b978b1b38"
server_key_iv: "ca740ca2544448c0005e9f2f7f513166"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.160727 4418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:42:34.161641 4440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.161844 4418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:34.161938 4418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "6dbf80e234e34512a63c317fef2d9992"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "0d9d5bdb48363c612336622b978b1b38"
server_key_iv: "ca740ca2544448c0005e9f2f7f513166"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.162009 4418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:34.180476 4418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:34.183058 4418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:34.183189 4418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:34.187770 4418 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:36425
I20260809 00:42:34.187801 4506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:36425 every 8 connection(s)
I20260809 00:42:34.188135 4418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:34.189303 4508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:34.191776 4508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992: Bootstrap starting.
I20260809 00:42:34.192065 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 4418
I20260809 00:42:34.192188 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:34.192379 31139 external_mini_cluster.cc:1428] Setting key 27b771f1621c164b091c4801bda13112
I20260809 00:42:34.192621 4508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:34.193012 4508 log.cc:824] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:34.193835 4508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992: No bootstrap required, opened a new log
I20260809 00:42:34.196189 4508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbf80e234e34512a63c317fef2d9992" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36425 } }
I20260809 00:42:34.196379 4508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:34.196425 4508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dbf80e234e34512a63c317fef2d9992, State: Initialized, Role: FOLLOWER
I20260809 00:42:34.196571 4508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbf80e234e34512a63c317fef2d9992" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36425 } }
I20260809 00:42:34.196650 4508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:34.196674 4508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:34.196743 4508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:34.197388 4508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbf80e234e34512a63c317fef2d9992" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36425 } }
I20260809 00:42:34.197499 4508 leader_election.cc:302] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dbf80e234e34512a63c317fef2d9992; no voters:
I20260809 00:42:34.197669 4508 leader_election.cc:288] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:34.197742 4513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:34.197917 4513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [term 1 LEADER]: Becoming Leader. State: Replica: 6dbf80e234e34512a63c317fef2d9992, State: Running, Role: LEADER
I20260809 00:42:34.197934 4508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:34.198014 4513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbf80e234e34512a63c317fef2d9992" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36425 } }
I20260809 00:42:34.198336 4513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dbf80e234e34512a63c317fef2d9992. Latest consensus state: current_term: 1 leader_uuid: "6dbf80e234e34512a63c317fef2d9992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbf80e234e34512a63c317fef2d9992" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36425 } } }
I20260809 00:42:34.198348 4515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dbf80e234e34512a63c317fef2d9992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dbf80e234e34512a63c317fef2d9992" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36425 } } }
I20260809 00:42:34.198822 4515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:34.198992 4513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:34.199133 4525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:34.199667 4525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:34.201229 4525 catalog_manager.cc:1347] Generated new cluster ID: da9e9f195f164465be28155e0f384e6e
I20260809 00:42:34.201304 4525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:34.205192 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.192961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.30.104.254:36425)
0809 00:42:34.193221 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.193233 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.193272 (+ 39us) server_negotiation.cc:407] Connection header received
0809 00:42:34.193861 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.193871 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.193939 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.194118 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:34.194819 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.196304 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.197262 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.197928 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.198437 (+ 509us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.201231 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:34.201236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:34.201236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:34.201249 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:34.203290 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.203743 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.203746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.203747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.203868 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.204084 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.204086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.204086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.204425 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:34.204607 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.204688 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.204956 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":95,"threads_started":1}
I20260809 00:42:34.210005 4525 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:34.210575 4525 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:34.215533 4525 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992: Generated new TSK 0
I20260809 00:42:34.215745 4525 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:34.250861 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36425
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:34.337286 4542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:34.337534 4542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:34.337561 4542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:34.338757 4542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:34.338799 4542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:34.338827 4542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:34.340444 4542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:34.340708 4542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:34.340956 4542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:34.346117 4550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:34.346128 4552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:34.346117 4548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:34.346372 4542 server_base.cc:1034] running on GCE node
I20260809 00:42:34.346637 4542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:34.346911 4542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:34.348054 4542 hybrid_clock.cc:648] HybridClock initialized: now 1786236154348037 us; error 37 us; skew 500 ppm
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:34.350659 4542 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:34.351079 4542 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:34.351123 4542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:34.351214 4542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:34.352322 4542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "688df95e4f9745d596e12f36abbe543f"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "25a6fb45f15f07640ce38c888e84789d"
server_key_iv: "a95847650ec17014cf151fe06463a875"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.352636 4542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "688df95e4f9745d596e12f36abbe543f"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "25a6fb45f15f07640ce38c888e84789d"
server_key_iv: "a95847650ec17014cf151fe06463a875"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.353904 4542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:42:34.354637 4559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.354831 4542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 00:42:34.354952 4542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "688df95e4f9745d596e12f36abbe543f"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "25a6fb45f15f07640ce38c888e84789d"
server_key_iv: "a95847650ec17014cf151fe06463a875"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.355049 4542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:34.371019 4542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:34.373270 4542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:34.373488 4542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:34.373767 4542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:34.374114 4542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:34.374168 4542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.374204 4542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:34.374248 4542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.380059 4542 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:32969
I20260809 00:42:34.380100 4715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:32969 every 8 connection(s)
I20260809 00:42:34.380381 4542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:34.387282 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 4542
I20260809 00:42:34.387405 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:34.387651 31139 external_mini_cluster.cc:1428] Setting key 0f8cd16fdb752d4e26c9a6a2a4ae52b7
I20260809 00:42:34.390611 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.381099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36425 (local address 127.30.104.193:38477)
0809 00:42:34.381271 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.381314 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:34.381801 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:34.382064 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:34.382067 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:34.382333 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:34.382837 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.382842 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.383593 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.383594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.384399 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.384401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.384859 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.384860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.384978 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.385560 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:34.385569 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:34.387035 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:34.388890 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:34.388898 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:34.388901 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:34.389251 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:34.389442 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:34.389443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:34.389443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:34.389596 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:34.389910 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:34.389917 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:34.390101 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 00:42:34.390219 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 00:42:34.390420 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260809 00:42:34.390671 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.381144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:38477 (local address 127.30.104.254:36425)
0809 00:42:34.381208 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.381210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.381398 (+ 188us) server_negotiation.cc:407] Connection header received
0809 00:42:34.381899 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.381900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.381910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.381996 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:34.382928 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.383510 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.384480 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.384683 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.384750 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.387150 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:34.387152 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:34.387153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:34.387160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:34.388809 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.389325 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.389325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.389326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.389381 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.389708 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.389710 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.389710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.389837 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:34.389901 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.390395 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.390537 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:42:34.391348 4716 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36425
I20260809 00:42:34.391492 4716 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:34.391652 4716 heartbeater.cc:511] Master 127.30.104.254:36425 requested a full tablet report, sending...
I20260809 00:42:34.392254 4465 ts_manager.cc:194] Registered new tserver with Master: 688df95e4f9745d596e12f36abbe543f (127.30.104.193:32969)
I20260809 00:42:34.393177 4465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:38477
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:34.415076 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36425
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:34.502272 4722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:34.502492 4722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:34.502518 4722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:34.503777 4722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:34.503822 4722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:34.503851 4722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:34.505403 4722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:34.505679 4722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:34.505915 4722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:34.511101 4727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:34.511193 4728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:34.511076 4731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:34.511343 4722 server_base.cc:1034] running on GCE node
I20260809 00:42:34.511627 4722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:34.511878 4722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:34.513053 4722 hybrid_clock.cc:648] HybridClock initialized: now 1786236154513029 us; error 40 us; skew 500 ppm
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:34.515738 4722 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:34.516101 4722 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:34.516152 4722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:34.516250 4722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:34.517329 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "03f0f9450fa04d7e8eb87d8d1853da06"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "a69f9afed9670789f5ba7f9eedb38daa"
server_key_iv: "f1b0a85f489ec357512d7be9219f2b3a"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.517654 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "03f0f9450fa04d7e8eb87d8d1853da06"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "a69f9afed9670789f5ba7f9eedb38daa"
server_key_iv: "f1b0a85f489ec357512d7be9219f2b3a"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.518980 4722 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 00:42:34.519757 4740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.519996 4722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:34.520078 4722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "03f0f9450fa04d7e8eb87d8d1853da06"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "a69f9afed9670789f5ba7f9eedb38daa"
server_key_iv: "f1b0a85f489ec357512d7be9219f2b3a"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.520136 4722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:34.529973 4722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:34.532224 4722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:34.532351 4722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:34.532557 4722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:34.532953 4722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:34.532990 4722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.533015 4722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:34.533061 4722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.538971 4722 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:46299
I20260809 00:42:34.539031 4902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:46299 every 8 connection(s)
I20260809 00:42:34.539326 4722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:34.541232 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 4722
I20260809 00:42:34.541366 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:34.541564 31139 external_mini_cluster.cc:1428] Setting key 8cb5b0d4f34d2da3df9055b4c799a780
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:34.549817 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.540170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:36459 (local address 127.30.104.254:36425)
0809 00:42:34.540238 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.540239 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.540413 (+ 174us) server_negotiation.cc:407] Connection header received
0809 00:42:34.540923 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.540924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.540934 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.541007 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:34.542152 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.542777 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.543934 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.544144 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.544235 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.546623 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:34.546624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:34.546625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:34.546633 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:34.548304 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.548778 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.548779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.548779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.548844 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.549129 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.549130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.549130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.549259 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:34.549330 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.549633 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.549722 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:42:34.550030 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.540127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36425 (local address 127.30.104.194:36459)
0809 00:42:34.540290 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.540333 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:34.540830 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:34.541065 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:34.541068 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:34.541336 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:34.542046 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.542053 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.542879 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.542881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.543831 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.543834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.544247 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.544248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.544372 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.544996 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:34.545005 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:34.546514 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:34.548419 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:34.548427 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:34.548430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:34.548685 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:34.548910 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:34.548910 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:34.548911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:34.549041 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:34.549331 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:34.549339 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:34.549487 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 00:42:34.549589 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 00:42:34.549770 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260809 00:42:34.550606 4903 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36425
I20260809 00:42:34.550726 4903 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:34.550886 4903 heartbeater.cc:511] Master 127.30.104.254:36425 requested a full tablet report, sending...
I20260809 00:42:34.551323 4465 ts_manager.cc:194] Registered new tserver with Master: 03f0f9450fa04d7e8eb87d8d1853da06 (127.30.104.194:46299)
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
I20260809 00:42:34.551964 4465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:36459
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:34.571080 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36425
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:42389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:34.655782 4909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:34.656008 4909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:34.656033 4909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:34.657248 4909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:34.657295 4909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:34.657322 4909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:34.658869 4909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:42389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:34.659088 4909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:34.659273 4909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:42:34.664829 4909 server_base.cc:1034] running on GCE node
W20260809 00:42:34.664791 4921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:34.664791 4918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:34.664798 4917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:34.665366 4909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:34.665660 4909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:34.666807 4909 hybrid_clock.cc:648] HybridClock initialized: now 1786236154666789 us; error 34 us; skew 500 ppm
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:34.669662 4909 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:34.670035 4909 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:34.670085 4909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:34.670215 4909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:34.671309 4909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "bdc0915f281a4bedb04823739d9eea46"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "a99fffe059a574771dace33829523eb1"
server_key_iv: "af2f2ae9a4e6d1bc1a4991a5d8423d0a"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.671739 4909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "bdc0915f281a4bedb04823739d9eea46"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "a99fffe059a574771dace33829523eb1"
server_key_iv: "af2f2ae9a4e6d1bc1a4991a5d8423d0a"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.673100 4909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:34.673888 4932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.674108 4909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:34.674206 4909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "bdc0915f281a4bedb04823739d9eea46"
format_stamp: "Formatted at 2026-08-09 00:42:34 on dist-test-slave-c12x"
server_key: "a99fffe059a574771dace33829523eb1"
server_key_iv: "af2f2ae9a4e6d1bc1a4991a5d8423d0a"
server_key_version: "encryptionkey@0"
I20260809 00:42:34.674278 4909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:34.681675 4909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:34.683957 4909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:34.684113 4909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:34.684370 4909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:34.684706 4909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:34.684741 4909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.684763 4909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:34.684777 4909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:34.690688 4909 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:36781
I20260809 00:42:34.690732 5100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:36781 every 8 connection(s)
I20260809 00:42:34.691002 4909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:34.697203 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 4909
I20260809 00:42:34.697307 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
Aug 09 00:42:34 dist-test-slave-c12x krb5kdc[4399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:34.697547 31139 external_mini_cluster.cc:1428] Setting key 83b5d5ca738f5e5d3786c9120378149b
I20260809 00:42:34.700865 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.691776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:60185 (local address 127.30.104.254:36425)
0809 00:42:34.691849 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.691852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.692103 (+ 251us) server_negotiation.cc:407] Connection header received
0809 00:42:34.692629 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.692630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.692644 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.692703 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:34.693764 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.694260 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.695394 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.695630 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.695729 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.698080 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:34.698081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:34.698081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:34.698089 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:34.699633 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.700026 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.700027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.700027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.700089 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.700313 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.700314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.700314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.700425 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:34.700479 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.700716 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.700777 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:34.701068 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.691740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36425 (local address 127.30.104.195:60185)
0809 00:42:34.691950 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.691988 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:34.692530 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:34.692821 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:34.692825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:34.693103 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:34.693676 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.693681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.694336 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.694337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.695284 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.695287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.695738 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.695739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.695845 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.696352 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:34.696360 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:34.697968 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:34.699704 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:34.699710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:34.699712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:34.699932 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:34.700156 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:34.700157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:34.700157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:34.700258 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:34.700491 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:34.700496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:34.700619 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 00:42:34.700699 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 00:42:34.700822 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":107,"threads_started":1}
I20260809 00:42:34.701521 5101 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36425
I20260809 00:42:34.701627 5101 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:34.701779 5101 heartbeater.cc:511] Master 127.30.104.254:36425 requested a full tablet report, sending...
I20260809 00:42:34.702163 4465 ts_manager.cc:194] Registered new tserver with Master: bdc0915f281a4bedb04823739d9eea46 (127.30.104.195:36781)
I20260809 00:42:34.702553 4465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:60185
I20260809 00:42:34.709558 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:42:34.717197 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.710920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.30.104.254:36425)
0809 00:42:34.711092 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.711095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.711109 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:42:34.711131 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.711131 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.711140 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.711187 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:34.711823 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.712324 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.713055 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.713231 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.713283 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.714520 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:34.714522 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:34.714522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:34.714529 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:34.716138 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.716474 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.716474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.716475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.716528 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:34.716700 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:34.716700 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:34.716701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:34.716850 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:34.716927 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.717030 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.717093 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260809 00:42:34.718978 4465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:34.720616 4465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:34.727643 5125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.723311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52018 (local address 127.30.104.195:36781)
0809 00:42:34.723490 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.723496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.723793 (+ 297us) server_negotiation.cc:407] Connection header received
0809 00:42:34.723867 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.723869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.723888 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.724069 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:34.724224 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.725387 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.726671 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.727164 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.727252 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.727311 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.727478 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.727534 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260809 00:42:34.727706 5120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.723302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42090 (local address 127.30.104.193:32969)
0809 00:42:34.723490 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.723495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.723986 (+ 491us) server_negotiation.cc:407] Connection header received
0809 00:42:34.724124 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.724127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.724142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.724231 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:34.724342 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.725030 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.726417 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.727191 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.727269 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.727328 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.727575 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.727619 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
I20260809 00:42:34.727720 5117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.723050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:36781 (local address 127.0.0.1:52018)
0809 00:42:34.723708 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.723717 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:34.723798 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:34.724003 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:34.724004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:34.724005 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:34.724169 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.724170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.725516 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.725517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.726587 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.726588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.727250 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.727251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.727331 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.727345 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:42:34.727516 (+ 171us) client_negotiation.cc:240] Negotiation successful
0809 00:42:34.727624 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":155,"thread_start_us":117,"threads_started":1}
I20260809 00:42:34.728127 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.723559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.30.104.194:46299)
0809 00:42:34.723747 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.723751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.723763 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:42:34.723805 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.723807 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.723824 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.724263 (+ 439us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:34.724342 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.725419 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.726584 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.727268 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.727675 (+ 407us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.727940 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.727978 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.728027 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260809 00:42:34.728210 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.723060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:32969 (local address 127.0.0.1:42090)
0809 00:42:34.723921 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.723930 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:34.724039 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:34.724229 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:34.724230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:34.724232 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:34.724281 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.724283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.725125 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.725126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.726327 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.726330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.727364 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.727366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.727468 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.727475 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 00:42:34.727527 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:42:34.728141 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":844,"mutex_wait_us":565,"thread_start_us":80,"threads_started":1}
I20260809 00:42:34.728268 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.723045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:46299 (local address 127.0.0.1:40056)
0809 00:42:34.723206 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.723246 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:34.723315 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:34.723987 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:34.723991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:34.723993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:34.724090 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.724093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.725553 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.725555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.726484 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.726487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.727366 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.727367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.727468 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.727476 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 00:42:34.727509 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 00:42:34.727595 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":3,"thread_start_us":56,"threads_started":1}
I20260809 00:42:34.728803 5005 tablet_service.cc:1467] Processing CreateTablet for tablet 0dad66fc68b3436b9ea1f58c7efd67bd (DEFAULT_TABLE table=test-table [id=f8e5af31738f4151b44c579f7d471eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:34.729250 5005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dad66fc68b3436b9ea1f58c7efd67bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:34.731901 5127 tablet_bootstrap.cc:492] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: Bootstrap starting.
I20260809 00:42:34.731945 4627 tablet_service.cc:1467] Processing CreateTablet for tablet 0dad66fc68b3436b9ea1f58c7efd67bd (DEFAULT_TABLE table=test-table [id=f8e5af31738f4151b44c579f7d471eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:34.732302 4627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dad66fc68b3436b9ea1f58c7efd67bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:34.732589 5127 tablet_bootstrap.cc:654] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:34.733100 5127 log.cc:824] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:34.733804 5127 tablet_bootstrap.cc:492] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: No bootstrap required, opened a new log
I20260809 00:42:34.733886 5127 ts_tablet_manager.cc:1397] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:34.735088 5130 tablet_bootstrap.cc:492] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f: Bootstrap starting.
I20260809 00:42:34.735500 4814 tablet_service.cc:1467] Processing CreateTablet for tablet 0dad66fc68b3436b9ea1f58c7efd67bd (DEFAULT_TABLE table=test-table [id=f8e5af31738f4151b44c579f7d471eb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:34.735826 5130 tablet_bootstrap.cc:654] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:34.735833 4814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dad66fc68b3436b9ea1f58c7efd67bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:34.735843 5127 raft_consensus.cc:348] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.736052 5127 raft_consensus.cc:374] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:34.736099 5127 raft_consensus.cc:729] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc0915f281a4bedb04823739d9eea46, State: Initialized, Role: FOLLOWER
I20260809 00:42:34.736239 5127 consensus_queue.cc:260] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.736464 5130 log.cc:824] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:34.736544 5127 ts_tablet_manager.cc:1428] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:42:34.736678 5101 heartbeater.cc:503] Master 127.30.104.254:36425 was elected leader, sending a full tablet report...
I20260809 00:42:34.737259 5130 tablet_bootstrap.cc:492] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f: No bootstrap required, opened a new log
I20260809 00:42:34.737308 5130 ts_tablet_manager.cc:1397] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:42:34.738634 5130 raft_consensus.cc:348] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.738761 5130 raft_consensus.cc:374] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:34.738782 5130 raft_consensus.cc:729] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688df95e4f9745d596e12f36abbe543f, State: Initialized, Role: FOLLOWER
I20260809 00:42:34.738919 5130 consensus_queue.cc:260] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.739215 5130 ts_tablet_manager.cc:1428] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:42:34.739202 5137 tablet_bootstrap.cc:492] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Bootstrap starting.
I20260809 00:42:34.739248 4716 heartbeater.cc:503] Master 127.30.104.254:36425 was elected leader, sending a full tablet report...
I20260809 00:42:34.739873 5137 tablet_bootstrap.cc:654] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:34.740375 5137 log.cc:824] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:34.741079 5137 tablet_bootstrap.cc:492] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: No bootstrap required, opened a new log
I20260809 00:42:34.741134 5137 ts_tablet_manager.cc:1397] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:42:34.742465 5137 raft_consensus.cc:348] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.742592 5137 raft_consensus.cc:374] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:34.742615 5137 raft_consensus.cc:729] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f0f9450fa04d7e8eb87d8d1853da06, State: Initialized, Role: FOLLOWER
I20260809 00:42:34.742731 5137 consensus_queue.cc:260] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.742993 5137 ts_tablet_manager.cc:1428] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:42:34.743042 4903 heartbeater.cc:503] Master 127.30.104.254:36425 was elected leader, sending a full tablet report...
W20260809 00:42:34.789930 4904 tablet.cc:2369] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:42:34.844776 5131 raft_consensus.cc:482] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:34.844950 5131 raft_consensus.cc:504] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.845377 5131 leader_election.cc:288] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03f0f9450fa04d7e8eb87d8d1853da06 (127.30.104.194:46299), 688df95e4f9745d596e12f36abbe543f (127.30.104.193:32969)
I20260809 00:42:34.848412 5126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.845636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:48461 (local address 127.30.104.194:46299)
0809 00:42:34.845687 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.845690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.845916 (+ 226us) server_negotiation.cc:407] Connection header received
0809 00:42:34.845947 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.845948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.845959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.846034 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:34.846201 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.846636 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.847657 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.848066 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.848124 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.848150 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.848294 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.848341 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:42:34.848429 5141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.845556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:46299 (local address 127.30.104.195:48461)
0809 00:42:34.845794 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.845804 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:34.845866 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:34.846059 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:34.846060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:34.846062 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:34.846131 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.846133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.846742 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.846742 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.847579 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.847581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.848154 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.848155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.848228 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.848231 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:34.848273 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 00:42:34.848309 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":42,"thread_start_us":118,"threads_started":1}
I20260809 00:42:34.848472 5120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.845675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:51195 (local address 127.30.104.193:32969)
0809 00:42:34.845722 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.845724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.845733 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:34.845766 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.845767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.845776 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.845848 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:34.846072 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.846617 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.847720 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.848084 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.848168 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.848176 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.848358 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.848398 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:42:34.848690 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.845555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:32969 (local address 127.30.104.195:51195)
0809 00:42:34.845616 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.845630 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:34.845687 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:34.845908 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:34.845910 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:34.845912 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:34.845993 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.845995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.846701 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.846703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.847610 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:34.847625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.848184 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.848185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:34.848280 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.848284 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:34.848341 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:42:34.848376 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":37}
I20260809 00:42:34.848842 4838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dad66fc68b3436b9ea1f58c7efd67bd" candidate_uuid: "bdc0915f281a4bedb04823739d9eea46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" is_pre_election: true
I20260809 00:42:34.849035 4838 raft_consensus.cc:2454] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdc0915f281a4bedb04823739d9eea46 in term 0.
I20260809 00:42:34.849009 4651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dad66fc68b3436b9ea1f58c7efd67bd" candidate_uuid: "bdc0915f281a4bedb04823739d9eea46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688df95e4f9745d596e12f36abbe543f" is_pre_election: true
I20260809 00:42:34.849171 4651 raft_consensus.cc:2454] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdc0915f281a4bedb04823739d9eea46 in term 0.
I20260809 00:42:34.849283 4934 leader_election.cc:302] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03f0f9450fa04d7e8eb87d8d1853da06, bdc0915f281a4bedb04823739d9eea46; no voters:
I20260809 00:42:34.849426 5131 raft_consensus.cc:2781] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:34.849484 5131 raft_consensus.cc:482] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:34.849519 5131 raft_consensus.cc:3037] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:34.850227 5131 raft_consensus.cc:504] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.850373 5131 leader_election.cc:288] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [CANDIDATE]: Term 1 election: Requested vote from peers 03f0f9450fa04d7e8eb87d8d1853da06 (127.30.104.194:46299), 688df95e4f9745d596e12f36abbe543f (127.30.104.193:32969)
I20260809 00:42:34.850551 4838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dad66fc68b3436b9ea1f58c7efd67bd" candidate_uuid: "bdc0915f281a4bedb04823739d9eea46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03f0f9450fa04d7e8eb87d8d1853da06"
I20260809 00:42:34.850575 4651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dad66fc68b3436b9ea1f58c7efd67bd" candidate_uuid: "bdc0915f281a4bedb04823739d9eea46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688df95e4f9745d596e12f36abbe543f"
I20260809 00:42:34.850625 4838 raft_consensus.cc:3037] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:34.850669 4651 raft_consensus.cc:3037] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:34.851341 4838 raft_consensus.cc:2454] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdc0915f281a4bedb04823739d9eea46 in term 1.
I20260809 00:42:34.851419 4651 raft_consensus.cc:2454] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdc0915f281a4bedb04823739d9eea46 in term 1.
I20260809 00:42:34.851523 4934 leader_election.cc:302] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03f0f9450fa04d7e8eb87d8d1853da06, bdc0915f281a4bedb04823739d9eea46; no voters:
I20260809 00:42:34.851627 5131 raft_consensus.cc:2781] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:34.851816 5131 raft_consensus.cc:686] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 1 LEADER]: Becoming Leader. State: Replica: bdc0915f281a4bedb04823739d9eea46, State: Running, Role: LEADER
I20260809 00:42:34.851895 5131 consensus_queue.cc:237] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } }
I20260809 00:42:34.852829 4465 catalog_manager.cc:5495] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 reported cstate change: term changed from 0 to 1, leader changed from <none> to bdc0915f281a4bedb04823739d9eea46 (127.30.104.195). New cstate: current_term: 1 leader_uuid: "bdc0915f281a4bedb04823739d9eea46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdc0915f281a4bedb04823739d9eea46" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:34.872570 5125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.870092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.30.104.195:36781)
0809 00:42:34.870150 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:34.870152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:34.870162 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:34.870202 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:34.870208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:34.870228 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:34.870304 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:34.870464 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.870935 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.871781 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:34.871979 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:34.872054 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:34.872217 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:34.872323 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:34.872416 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:34.872436 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 00:42:34.872456 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:42:34.874524 4651 raft_consensus.cc:1261] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 1 FOLLOWER]: Refusing update from remote peer bdc0915f281a4bedb04823739d9eea46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:34.874547 4838 raft_consensus.cc:1261] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 1 FOLLOWER]: Refusing update from remote peer bdc0915f281a4bedb04823739d9eea46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:34.874900 5131 consensus_queue.cc:1035] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688df95e4f9745d596e12f36abbe543f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:34.875062 5142 consensus_queue.cc:1035] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03f0f9450fa04d7e8eb87d8d1853da06" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:34.877686 5147 mvcc.cc:205] Tried to move back new op lower bound from 7316423290363011072 to 7316423290274312192. Current Snapshot: MvccSnapshot[applied={T|T < 7316423290363011072}]
I20260809 00:42:34.880650 4464 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:34.880757 4464 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:34.881564 4464 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6dbf80e234e34512a63c317fef2d9992: Sending DeleteTablet for 3 replicas of tablet 0dad66fc68b3436b9ea1f58c7efd67bd
I20260809 00:42:34.881888 4814 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dad66fc68b3436b9ea1f58c7efd67bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:34 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:40056
I20260809 00:42:34.882002 4627 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dad66fc68b3436b9ea1f58c7efd67bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:34 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:42090
I20260809 00:42:34.882102 5005 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dad66fc68b3436b9ea1f58c7efd67bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:34 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:52018
I20260809 00:42:34.882256 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 4542
I20260809 00:42:34.882272 5167 tablet_replica.cc:330] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f: stopping tablet replica
I20260809 00:42:34.882330 5164 tablet_replica.cc:330] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: stopping tablet replica
I20260809 00:42:34.882480 5167 raft_consensus.cc:2229] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:34.882624 5167 raft_consensus.cc:2258] T 0dad66fc68b3436b9ea1f58c7efd67bd P 688df95e4f9745d596e12f36abbe543f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:34.882997 5164 raft_consensus.cc:2229] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:34.883128 5164 raft_consensus.cc:2258] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:34.886294 5164 ts_tablet_manager.cc:1905] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:34.888723 5164 ts_tablet_manager.cc:1918] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:34.888834 5164 log.cc:1196] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/0dad66fc68b3436b9ea1f58c7efd67bd
I20260809 00:42:34.889249 5164 ts_tablet_manager.cc:1939] T 0dad66fc68b3436b9ea1f58c7efd67bd P 03f0f9450fa04d7e8eb87d8d1853da06: Deleting consensus metadata
I20260809 00:42:34.892045 4443 catalog_manager.cc:4841] TS 03f0f9450fa04d7e8eb87d8d1853da06 (127.30.104.194:46299): tablet 0dad66fc68b3436b9ea1f58c7efd67bd (table test-table [id=f8e5af31738f4151b44c579f7d471eb8]) successfully deleted
I20260809 00:42:34.892482 5168 tablet_replica.cc:330] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: stopping tablet replica
I20260809 00:42:34.892609 5168 raft_consensus.cc:2229] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:34.892751 5168 raft_consensus.cc:2258] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:34.893155 5168 ts_tablet_manager.cc:1905] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:34.894479 5168 ts_tablet_manager.cc:1918] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:34.894539 5168 log.cc:1196] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/0dad66fc68b3436b9ea1f58c7efd67bd
I20260809 00:42:34.894796 5168 ts_tablet_manager.cc:1939] T 0dad66fc68b3436b9ea1f58c7efd67bd P bdc0915f281a4bedb04823739d9eea46: Deleting consensus metadata
I20260809 00:42:34.895239 4445 catalog_manager.cc:4841] TS bdc0915f281a4bedb04823739d9eea46 (127.30.104.195:36781): tablet 0dad66fc68b3436b9ea1f58c7efd67bd (table test-table [id=f8e5af31738f4151b44c579f7d471eb8]) successfully deleted
W20260809 00:42:34.896904 4446 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:32969 (error 108)
I20260809 00:42:34.897192 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 4722
I20260809 00:42:34.899057 5116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:34.897358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:32969 (local address 127.0.0.1:42094)
0809 00:42:34.898956 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:34.899013 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:32969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1568}
W20260809 00:42:34.899971 4446 catalog_manager.cc:4584] TS 688df95e4f9745d596e12f36abbe543f (127.30.104.193:32969): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0dad66fc68b3436b9ea1f58c7efd67bd: Network error: Client connection negotiation failed: client connection to 127.30.104.193:32969: connect: Connection refused (error 111)
I20260809 00:42:34.909966 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 4909
I20260809 00:42:34.922945 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 4418
2026-08-09T00:42:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2233 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T00:42:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:34Z Disabled control of system clock
I20260809 00:42:34.958218 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:34973
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:34973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:42:35.044222 5176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:35.044443 5176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:35.044466 5176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:35.045655 5176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:35.045706 5176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:35.045720 5176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:35.045733 5176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:35.045746 5176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:35.047151 5176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:34973
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:34973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:35.047415 5176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:35.047736 5176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:35.052894 5186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.052943 5185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.052966 5189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:35.053208 5176 server_base.cc:1034] running on GCE node
I20260809 00:42:35.053514 5176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:35.053817 5176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:35.055006 5176 hybrid_clock.cc:648] HybridClock initialized: now 1786236155054957 us; error 72 us; skew 500 ppm
I20260809 00:42:35.056571 5176 webserver.cc:460] Webserver started at http://127.30.104.254:44751/ using document root <none> and password file <none>
I20260809 00:42:35.056851 5176 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:35.056926 5176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:35.057094 5176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:35.058295 5176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "b5d1a955d2bb402bad30d3f44985728b"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "089cdd2519bc94bbc133aadbc8940235"
server_key_iv: "66deaa3d294a193d574dc5072db3d65e"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.058722 5176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "b5d1a955d2bb402bad30d3f44985728b"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "089cdd2519bc94bbc133aadbc8940235"
server_key_iv: "66deaa3d294a193d574dc5072db3d65e"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.060256 5176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:35.061193 5199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.061408 5176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:35.061520 5176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "b5d1a955d2bb402bad30d3f44985728b"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "089cdd2519bc94bbc133aadbc8940235"
server_key_iv: "66deaa3d294a193d574dc5072db3d65e"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.061601 5176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:35.065531 5176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:35.065832 5176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:35.065986 5176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:35.070369 5176 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:34973
I20260809 00:42:35.070425 5274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:34973 every 8 connection(s)
I20260809 00:42:35.070808 5176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:35.071635 5276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:35.073148 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 5176
I20260809 00:42:35.073277 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:35.073463 31139 external_mini_cluster.cc:1428] Setting key 22b6f70f3396be91eb1980f1e2be281f
I20260809 00:42:35.074883 5276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b: Bootstrap starting.
I20260809 00:42:35.075641 5276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:35.075946 5276 log.cc:824] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:35.076280 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.074039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42312 (local address 127.30.104.254:34973)
0809 00:42:35.074244 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.074252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.074287 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:42:35.074814 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.074822 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.074877 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.075045 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.075796 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.075800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.075800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.075805 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.075849 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.075859 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.075940 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.076053 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.076118 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
I20260809 00:42:35.076640 5276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b: No bootstrap required, opened a new log
I20260809 00:42:35.078091 5276 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d1a955d2bb402bad30d3f44985728b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34973 } }
I20260809 00:42:35.078258 5276 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:35.078298 5276 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5d1a955d2bb402bad30d3f44985728b, State: Initialized, Role: FOLLOWER
I20260809 00:42:35.078475 5276 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d1a955d2bb402bad30d3f44985728b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34973 } }
I20260809 00:42:35.078590 5276 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:35.078650 5276 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:35.078703 5276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:35.079586 5276 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d1a955d2bb402bad30d3f44985728b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34973 } }
I20260809 00:42:35.079680 5276 leader_election.cc:302] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5d1a955d2bb402bad30d3f44985728b; no voters:
I20260809 00:42:35.079838 5276 leader_election.cc:288] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:35.079878 5282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:35.080009 5282 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [term 1 LEADER]: Becoming Leader. State: Replica: b5d1a955d2bb402bad30d3f44985728b, State: Running, Role: LEADER
I20260809 00:42:35.080122 5276 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:35.080116 5282 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d1a955d2bb402bad30d3f44985728b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34973 } }
I20260809 00:42:35.080536 5283 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5d1a955d2bb402bad30d3f44985728b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d1a955d2bb402bad30d3f44985728b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34973 } } }
I20260809 00:42:35.080536 5284 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5d1a955d2bb402bad30d3f44985728b. Latest consensus state: current_term: 1 leader_uuid: "b5d1a955d2bb402bad30d3f44985728b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5d1a955d2bb402bad30d3f44985728b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34973 } } }
I20260809 00:42:35.080672 5283 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:35.080678 5284 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:35.080924 5292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:35.081458 5292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:35.083125 5292 catalog_manager.cc:1347] Generated new cluster ID: 8ad5578fbe854e7a907f06ac34a3ea7b
I20260809 00:42:35.083189 5292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:35.089125 5292 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:35.089280 5292 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:35.094110 5292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b: Generated new TSK 0
I20260809 00:42:35.094415 5292 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:42:35.101245 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:34973
--builtin_ntp_servers=127.30.104.212:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:42:35.203097 5307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:35.203338 5307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:35.203379 5307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:35.204633 5307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:35.204708 5307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:35.204756 5307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:35.206177 5307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:35.206451 5307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:35.206765 5307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:42:35.213275 5307 server_base.cc:1034] running on GCE node
W20260809 00:42:35.213219 5322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.213238 5316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.213451 5315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:35.213680 5307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:35.214020 5307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:35.215219 5307 hybrid_clock.cc:648] HybridClock initialized: now 1786236155215205 us; error 31 us; skew 500 ppm
I20260809 00:42:35.217744 5307 webserver.cc:460] Webserver started at http://127.30.104.193:41909/ using document root <none> and password file <none>
I20260809 00:42:35.218061 5307 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:35.218122 5307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:35.218253 5307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:35.219702 5307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "3194166c3f0d40c8ab9732c8bb1ed124"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "524ec5a6633422223c47339d9d29868a"
server_key_iv: "1177172c87ac03ff163f633309591899"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.220130 5307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "3194166c3f0d40c8ab9732c8bb1ed124"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "524ec5a6633422223c47339d9d29868a"
server_key_iv: "1177172c87ac03ff163f633309591899"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.221777 5307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:35.222776 5333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.222967 5307 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:35.223083 5307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "3194166c3f0d40c8ab9732c8bb1ed124"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "524ec5a6633422223c47339d9d29868a"
server_key_iv: "1177172c87ac03ff163f633309591899"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.223171 5307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:35.226573 5307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:35.226861 5307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:35.227002 5307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:35.227240 5307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:35.227612 5307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:35.227648 5307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.227669 5307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:35.227682 5307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.233650 5307 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:33195
I20260809 00:42:35.233775 5490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:33195 every 8 connection(s)
I20260809 00:42:35.234078 5307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:35.237710 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.234954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:53153 (local address 127.30.104.254:34973)
0809 00:42:35.235028 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.235030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.235313 (+ 283us) server_negotiation.cc:407] Connection header received
0809 00:42:35.236035 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.236037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.236043 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.236120 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.237450 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.237452 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.237452 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.237456 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.237472 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.237476 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.237545 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.237638 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.237651 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:35.237895 5494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.234955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:34973 (local address 127.30.104.193:53153)
0809 00:42:35.235179 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.235219 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.235899 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.236193 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.236196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.236206 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:35.237270 (+ 1064us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:42:35.237271 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:35.237308 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:42:35.237340 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:35.237554 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:35.237560 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:35.237563 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:35.237639 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 00:42:35.237702 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":100,"threads_started":1}
I20260809 00:42:35.238406 5492 heartbeater.cc:347] Connected to a master server at 127.30.104.254:34973
I20260809 00:42:35.238524 5492 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:35.238677 5492 heartbeater.cc:511] Master 127.30.104.254:34973 requested a full tablet report, sending...
I20260809 00:42:35.239108 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 5307
I20260809 00:42:35.239230 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:35.239295 5222 ts_manager.cc:194] Registered new tserver with Master: 3194166c3f0d40c8ab9732c8bb1ed124 (127.30.104.193:33195)
I20260809 00:42:35.239439 31139 external_mini_cluster.cc:1428] Setting key 7864ef8c491e0808166d19b7b703aca0
I20260809 00:42:35.240402 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:34973
--builtin_ntp_servers=127.30.104.212:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:42:35.328783 5495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:35.329051 5495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:35.329115 5495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:35.330320 5495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:35.330391 5495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:35.330438 5495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:35.331926 5495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:35.332227 5495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:35.332566 5495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:35.338016 5507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.337996 5503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.337996 5505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:35.338656 5495 server_base.cc:1034] running on GCE node
I20260809 00:42:35.338898 5495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:35.339123 5495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:35.340289 5495 hybrid_clock.cc:648] HybridClock initialized: now 1786236155340271 us; error 30 us; skew 500 ppm
I20260809 00:42:35.341822 5495 webserver.cc:460] Webserver started at http://127.30.104.194:37041/ using document root <none> and password file <none>
I20260809 00:42:35.342053 5495 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:35.342096 5495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:35.342188 5495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:35.343245 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "5cf61560a7974f8f9ad59809f9c42d57"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "5e7d231b646b34fcb6f4a08bd4581580"
server_key_iv: "60f9f37a6d39bc70763042e51a954e0c"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.343652 5495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "5cf61560a7974f8f9ad59809f9c42d57"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "5e7d231b646b34fcb6f4a08bd4581580"
server_key_iv: "60f9f37a6d39bc70763042e51a954e0c"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.344937 5495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:35.345623 5516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.345844 5495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:35.345937 5495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "5cf61560a7974f8f9ad59809f9c42d57"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "5e7d231b646b34fcb6f4a08bd4581580"
server_key_iv: "60f9f37a6d39bc70763042e51a954e0c"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.345993 5495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:35.349469 5495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:35.349740 5495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:35.349839 5495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:35.350008 5495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:35.350327 5495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:35.350359 5495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.350389 5495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:35.350435 5495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.357492 5495 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:35755
I20260809 00:42:35.357622 5667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:35755 every 8 connection(s)
I20260809 00:42:35.358078 5495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:35.363119 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.360271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:57937 (local address 127.30.104.254:34973)
0809 00:42:35.360341 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.360343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.360630 (+ 287us) server_negotiation.cc:407] Connection header received
0809 00:42:35.361302 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.361304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.361311 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.361364 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.362835 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.362837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.362837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.362843 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.362859 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.362864 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.362918 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.363023 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.363040 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:42:35.363322 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.360276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:34973 (local address 127.30.104.194:57937)
0809 00:42:35.360492 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.360535 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.361171 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.361435 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.361440 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.361449 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:35.362670 (+ 1221us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:42:35.362670 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:35.362701 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:42:35.362733 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:35.362936 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:35.362941 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:35.362944 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:35.363033 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:42:35.363100 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
I20260809 00:42:35.363888 5668 heartbeater.cc:347] Connected to a master server at 127.30.104.254:34973
I20260809 00:42:35.363981 5668 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:35.364162 5668 heartbeater.cc:511] Master 127.30.104.254:34973 requested a full tablet report, sending...
I20260809 00:42:35.364540 5222 ts_manager.cc:194] Registered new tserver with Master: 5cf61560a7974f8f9ad59809f9c42d57 (127.30.104.194:35755)
I20260809 00:42:35.366225 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 5495
I20260809 00:42:35.366310 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:35.366487 31139 external_mini_cluster.cc:1428] Setting key 745709314e411ed69cde8aa1fe723faa
I20260809 00:42:35.367712 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:34973
--builtin_ntp_servers=127.30.104.212:35883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 00:42:35.454527 5671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:35.454743 5671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:35.454766 5671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:35.455926 5671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:35.455968 5671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:35.455993 5671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:35.457361 5671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:34973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:35.457569 5671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:35.457823 5671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:35.462908 5682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.462975 5681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.462908 5684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:35.463140 5671 server_base.cc:1034] running on GCE node
I20260809 00:42:35.463374 5671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:35.463717 5671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:35.464905 5671 hybrid_clock.cc:648] HybridClock initialized: now 1786236155464898 us; error 55 us; skew 500 ppm
I20260809 00:42:35.466382 5671 webserver.cc:460] Webserver started at http://127.30.104.195:40375/ using document root <none> and password file <none>
I20260809 00:42:35.466686 5671 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:35.466763 5671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:35.466928 5671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:35.468174 5671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "8a8ceff2a43443f7a1226dd0e23f3161"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "d0256d80ef06e45ac34d9a34e271eb06"
server_key_iv: "c50f10d1db236f6ead6fe3b523dac68b"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.468652 5671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "8a8ceff2a43443f7a1226dd0e23f3161"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "d0256d80ef06e45ac34d9a34e271eb06"
server_key_iv: "c50f10d1db236f6ead6fe3b523dac68b"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.470111 5671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:35.471006 5694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.471240 5671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:35.471357 5671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "8a8ceff2a43443f7a1226dd0e23f3161"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "d0256d80ef06e45ac34d9a34e271eb06"
server_key_iv: "c50f10d1db236f6ead6fe3b523dac68b"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.471438 5671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:35.474980 5671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:35.475265 5671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:35.475417 5671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:35.475672 5671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:35.476014 5671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:35.476073 5671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.476114 5671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:35.476140 5671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.482187 5671 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:38995
I20260809 00:42:35.482235 5846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:38995 every 8 connection(s)
I20260809 00:42:35.482635 5671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:35.483677 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 5671
I20260809 00:42:35.483798 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:35.483980 31139 external_mini_cluster.cc:1428] Setting key fa0f47aac52cce70e967b01ec85bc12c
I20260809 00:42:35.485924 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.483508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:59293 (local address 127.30.104.254:34973)
0809 00:42:35.483581 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.483583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.483783 (+ 200us) server_negotiation.cc:407] Connection header received
0809 00:42:35.484440 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.484441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.484447 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.484495 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.485684 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.485686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.485686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.485692 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.485707 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.485712 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.485763 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.485838 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.485853 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:35.486135 5849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.483443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:34973 (local address 127.30.104.195:59293)
0809 00:42:35.483661 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.483703 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.484334 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.484567 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.484571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.484581 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:35.485530 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:42:35.485531 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:35.485560 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:42:35.485589 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:35.485761 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:35.485765 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:35.485769 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:35.485848 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:42:35.485909 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260809 00:42:35.486618 5847 heartbeater.cc:347] Connected to a master server at 127.30.104.254:34973
I20260809 00:42:35.486742 5847 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:35.486896 5847 heartbeater.cc:511] Master 127.30.104.254:34973 requested a full tablet report, sending...
I20260809 00:42:35.487250 5222 ts_manager.cc:194] Registered new tserver with Master: 8a8ceff2a43443f7a1226dd0e23f3161 (127.30.104.195:38995)
I20260809 00:42:35.496088 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:42:35.498787 5280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.497441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42318 (local address 127.30.104.254:34973)
0809 00:42:35.497548 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.497550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.497620 (+ 70us) server_negotiation.cc:407] Connection header received
0809 00:42:35.497661 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.497662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.497667 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.497719 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.498483 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.498492 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.498493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.498497 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.498513 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.498517 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.498575 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.498633 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.498645 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 00:42:35.500816 5222 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:35.502455 5222 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:35.506456 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.505008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33195 (local address 127.0.0.1:35316)
0809 00:42:35.505170 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.505178 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.505209 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.505561 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.505562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.505564 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:35.506042 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:42:35.506043 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:35.506047 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:42:35.506055 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:35.506285 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:35.506292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:35.506296 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:35.506369 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:42:35.506384 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260809 00:42:35.506471 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.505197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35316 (local address 127.30.104.193:33195)
0809 00:42:35.505408 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.505412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.505422 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:35.505450 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.505453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.505463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.505542 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.506139 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.506147 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.506147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.506152 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.506194 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.506214 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.506291 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.506379 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.506395 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260809 00:42:35.506865 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.504914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.30.104.195:38995)
0809 00:42:35.505097 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.505103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.505228 (+ 125us) server_negotiation.cc:407] Connection header received
0809 00:42:35.505258 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.505261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.505271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.505352 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.506151 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.506155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.506155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.506159 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.506189 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.506209 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.506260 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.506736 (+ 476us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.506755 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260809 00:42:35.507046 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.504785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:38995 (local address 127.0.0.1:59808)
0809 00:42:35.505111 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.505121 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.505167 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.505372 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.505386 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.505392 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:35.505895 (+ 503us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:42:35.505896 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:35.505912 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:42:35.505937 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:35.506646 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:35.506647 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:35.506648 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:42:35.506959 (+ 311us) client_negotiation.cc:240] Negotiation successful
0809 00:42:35.506974 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":110,"threads_started":1}
I20260809 00:42:35.507341 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.504757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:35755 (local address 127.0.0.1:39908)
0809 00:42:35.504965 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.505006 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.505081 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.506466 (+ 1385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.506467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.506470 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:35.506944 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:42:35.506945 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:35.506953 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:42:35.506969 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:35.507223 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:35.507224 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:35.507225 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:42:35.507257 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 00:42:35.507270 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260809 00:42:35.507512 5399 tablet_service.cc:1467] Processing CreateTablet for tablet 7a39c6ec2bc646a2b344279bbd6184a1 (DEFAULT_TABLE table=test-table [id=ca5554a6dc5b487286fac5928efebc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:35.507730 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.504857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39908 (local address 127.30.104.194:35755)
0809 00:42:35.506334 (+ 1477us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.506338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.506345 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:42:35.506368 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.506369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.506375 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.506461 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.507067 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.507076 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.507077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.507081 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.507115 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.507143 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.507576 (+ 433us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.507601 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.507618 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1444,"thread_start_us":89,"threads_started":1}
I20260809 00:42:35.507764 5759 tablet_service.cc:1467] Processing CreateTablet for tablet 7a39c6ec2bc646a2b344279bbd6184a1 (DEFAULT_TABLE table=test-table [id=ca5554a6dc5b487286fac5928efebc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:35.507937 5399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a39c6ec2bc646a2b344279bbd6184a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:35.508060 5759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a39c6ec2bc646a2b344279bbd6184a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:35.508394 5562 tablet_service.cc:1467] Processing CreateTablet for tablet 7a39c6ec2bc646a2b344279bbd6184a1 (DEFAULT_TABLE table=test-table [id=ca5554a6dc5b487286fac5928efebc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:35.508718 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a39c6ec2bc646a2b344279bbd6184a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:35.511555 5867 tablet_bootstrap.cc:492] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: Bootstrap starting.
I20260809 00:42:35.511626 5869 tablet_bootstrap.cc:492] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57: Bootstrap starting.
I20260809 00:42:35.512308 5867 tablet_bootstrap.cc:654] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:35.512308 5869 tablet_bootstrap.cc:654] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:35.512660 5867 log.cc:824] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:35.512660 5869 log.cc:824] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:35.512995 5866 tablet_bootstrap.cc:492] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124: Bootstrap starting.
I20260809 00:42:35.513482 5867 tablet_bootstrap.cc:492] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: No bootstrap required, opened a new log
I20260809 00:42:35.513612 5867 ts_tablet_manager.cc:1397] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:35.513723 5869 tablet_bootstrap.cc:492] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57: No bootstrap required, opened a new log
I20260809 00:42:35.513780 5866 tablet_bootstrap.cc:654] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:35.513803 5869 ts_tablet_manager.cc:1397] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:35.514107 5866 log.cc:824] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:35.514945 5866 tablet_bootstrap.cc:492] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124: No bootstrap required, opened a new log
I20260809 00:42:35.515039 5866 ts_tablet_manager.cc:1397] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:35.515944 5867 raft_consensus.cc:348] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.516109 5867 raft_consensus.cc:374] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:35.516155 5867 raft_consensus.cc:729] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a8ceff2a43443f7a1226dd0e23f3161, State: Initialized, Role: FOLLOWER
I20260809 00:42:35.516292 5867 consensus_queue.cc:260] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.516554 5867 ts_tablet_manager.cc:1428] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:42:35.516482 5869 raft_consensus.cc:348] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.516633 5869 raft_consensus.cc:374] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:35.516675 5869 raft_consensus.cc:729] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf61560a7974f8f9ad59809f9c42d57, State: Initialized, Role: FOLLOWER
I20260809 00:42:35.516708 5847 heartbeater.cc:503] Master 127.30.104.254:34973 was elected leader, sending a full tablet report...
I20260809 00:42:35.516800 5869 consensus_queue.cc:260] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.517081 5869 ts_tablet_manager.cc:1428] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:42:35.517189 5668 heartbeater.cc:503] Master 127.30.104.254:34973 was elected leader, sending a full tablet report...
I20260809 00:42:35.518180 5866 raft_consensus.cc:348] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.518302 5866 raft_consensus.cc:374] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:35.518322 5866 raft_consensus.cc:729] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3194166c3f0d40c8ab9732c8bb1ed124, State: Initialized, Role: FOLLOWER
I20260809 00:42:35.518429 5866 consensus_queue.cc:260] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.518641 5866 ts_tablet_manager.cc:1428] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 00:42:35.518714 5492 heartbeater.cc:503] Master 127.30.104.254:34973 was elected leader, sending a full tablet report...
I20260809 00:42:35.583853 5874 raft_consensus.cc:482] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:35.583986 5874 raft_consensus.cc:504] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.584318 5874 leader_election.cc:288] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cf61560a7974f8f9ad59809f9c42d57 (127.30.104.194:35755), 3194166c3f0d40c8ab9732c8bb1ed124 (127.30.104.193:33195)
I20260809 00:42:35.585723 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.584704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:35433 (local address 127.30.104.193:33195)
0809 00:42:35.584748 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.584751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.584761 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:35.584797 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.584798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.584803 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.584853 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.585467 (+ 614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.585469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.585469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.585475 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.585492 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.585497 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.585556 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.585639 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.585654 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:35.585723 5849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.584537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33195 (local address 127.30.104.195:35433)
0809 00:42:35.584605 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.584621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.584688 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.584881 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.584882 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.584885 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:35.585351 (+ 466us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:42:35.585351 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:35.585359 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:42:35.585371 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:35.585557 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:35.585558 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:35.585559 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:42:35.585636 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:42:35.585651 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 00:42:35.585872 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.584660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:57715 (local address 127.30.104.194:35755)
0809 00:42:35.584748 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.584751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.584932 (+ 181us) server_negotiation.cc:407] Connection header received
0809 00:42:35.584956 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.584957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.584962 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.585001 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.585633 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.585635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.585635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.585640 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.585662 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.585666 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.585729 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.585787 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.585801 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 00:42:35.585896 5877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.584537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:35755 (local address 127.30.104.195:57715)
0809 00:42:35.584801 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.584809 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.584888 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.585028 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.585028 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.585044 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:35.585513 (+ 469us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:42:35.585514 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:35.585522 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:42:35.585537 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:35.585727 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:35.585727 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:35.585728 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 00:42:35.585787 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:42:35.585801 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":71,"thread_start_us":103,"threads_started":1}
I20260809 00:42:35.586100 5424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a39c6ec2bc646a2b344279bbd6184a1" candidate_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" is_pre_election: true
I20260809 00:42:35.586274 5424 raft_consensus.cc:2454] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a8ceff2a43443f7a1226dd0e23f3161 in term 0.
I20260809 00:42:35.586535 5700 leader_election.cc:302] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3194166c3f0d40c8ab9732c8bb1ed124, 8a8ceff2a43443f7a1226dd0e23f3161; no voters:
I20260809 00:42:35.586724 5874 raft_consensus.cc:2781] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:35.586789 5874 raft_consensus.cc:482] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:35.586814 5874 raft_consensus.cc:3037] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:35.586750 5607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a39c6ec2bc646a2b344279bbd6184a1" candidate_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf61560a7974f8f9ad59809f9c42d57" is_pre_election: true
I20260809 00:42:35.586903 5607 raft_consensus.cc:2454] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a8ceff2a43443f7a1226dd0e23f3161 in term 0.
I20260809 00:42:35.587738 5874 raft_consensus.cc:504] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.587954 5874 leader_election.cc:288] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [CANDIDATE]: Term 1 election: Requested vote from peers 5cf61560a7974f8f9ad59809f9c42d57 (127.30.104.194:35755), 3194166c3f0d40c8ab9732c8bb1ed124 (127.30.104.193:33195)
I20260809 00:42:35.588056 5607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a39c6ec2bc646a2b344279bbd6184a1" candidate_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf61560a7974f8f9ad59809f9c42d57"
I20260809 00:42:35.588070 5424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a39c6ec2bc646a2b344279bbd6184a1" candidate_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3194166c3f0d40c8ab9732c8bb1ed124"
I20260809 00:42:35.588142 5607 raft_consensus.cc:3037] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:35.588157 5424 raft_consensus.cc:3037] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:35.588873 5607 raft_consensus.cc:2454] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a8ceff2a43443f7a1226dd0e23f3161 in term 1.
I20260809 00:42:35.588893 5424 raft_consensus.cc:2454] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a8ceff2a43443f7a1226dd0e23f3161 in term 1.
I20260809 00:42:35.589084 5700 leader_election.cc:302] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3194166c3f0d40c8ab9732c8bb1ed124, 8a8ceff2a43443f7a1226dd0e23f3161; no voters:
I20260809 00:42:35.589212 5874 raft_consensus.cc:2781] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:35.589376 5874 raft_consensus.cc:686] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 1 LEADER]: Becoming Leader. State: Replica: 8a8ceff2a43443f7a1226dd0e23f3161, State: Running, Role: LEADER
I20260809 00:42:35.589515 5874 consensus_queue.cc:237] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } }
I20260809 00:42:35.590432 5221 catalog_manager.cc:5495] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a8ceff2a43443f7a1226dd0e23f3161 (127.30.104.195). New cstate: current_term: 1 leader_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a8ceff2a43443f7a1226dd0e23f3161" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 38995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:35.602283 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.601151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59820 (local address 127.30.104.195:38995)
0809 00:42:35.601221 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.601223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.601232 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:35.601271 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.601272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.601285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.601335 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.601971 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.601972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.601973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.601977 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.601993 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.601997 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.602047 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.602154 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.602167 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:42:35.604416 5424 raft_consensus.cc:1261] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124 [term 1 FOLLOWER]: Refusing update from remote peer 8a8ceff2a43443f7a1226dd0e23f3161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:35.604506 5607 raft_consensus.cc:1261] T 7a39c6ec2bc646a2b344279bbd6184a1 P 5cf61560a7974f8f9ad59809f9c42d57 [term 1 FOLLOWER]: Refusing update from remote peer 8a8ceff2a43443f7a1226dd0e23f3161: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:35.604836 5879 consensus_queue.cc:1035] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cf61560a7974f8f9ad59809f9c42d57" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 35755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:35.604975 5874 consensus_queue.cc:1035] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3194166c3f0d40c8ab9732c8bb1ed124" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:35.608299 5887 mvcc.cc:205] Tried to move back new op lower bound from 7316423293352931328 to 7316423293295525888. Current Snapshot: MvccSnapshot[applied={T|T < 7316423293352931328 or (T in {7316423293352931328})}]
I20260809 00:42:35.610908 5222 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:35.611040 5222 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:35.612033 5222 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b5d1a955d2bb402bad30d3f44985728b: Sending DeleteTablet for 3 replicas of tablet 7a39c6ec2bc646a2b344279bbd6184a1
I20260809 00:42:35.612442 5562 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a39c6ec2bc646a2b344279bbd6184a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:35 UTC) from {username='slave'} at 127.0.0.1:39908
I20260809 00:42:35.612449 5399 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a39c6ec2bc646a2b344279bbd6184a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:35 UTC) from {username='slave'} at 127.0.0.1:35316
I20260809 00:42:35.612449 5759 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a39c6ec2bc646a2b344279bbd6184a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:35 UTC) from {username='slave'} at 127.0.0.1:59808
I20260809 00:42:35.612681 5896 tablet_replica.cc:330] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: stopping tablet replica
I20260809 00:42:35.612783 5896 raft_consensus.cc:2229] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:35.612892 5896 raft_consensus.cc:2258] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:35.612916 5897 tablet_replica.cc:330] T 7a39c6ec2bc646a2b344279bbd6184a1 P 3194166c3f0d40c8ab9732c8bb1ed124: stopping tablet replica
I20260809 00:42:35.613088 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 5307
I20260809 00:42:35.615711 5896 ts_tablet_manager.cc:1905] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:35.617527 5896 ts_tablet_manager.cc:1918] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:35.617604 5896 log.cc:1196] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/7a39c6ec2bc646a2b344279bbd6184a1
I20260809 00:42:35.617913 5896 ts_tablet_manager.cc:1939] T 7a39c6ec2bc646a2b344279bbd6184a1 P 8a8ceff2a43443f7a1226dd0e23f3161: Deleting consensus metadata
I20260809 00:42:35.619592 5203 catalog_manager.cc:4841] TS 8a8ceff2a43443f7a1226dd0e23f3161 (127.30.104.195:38995): tablet 7a39c6ec2bc646a2b344279bbd6184a1 (table test-table [id=ca5554a6dc5b487286fac5928efebc48]) successfully deleted
W20260809 00:42:35.625794 5203 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:33195 (error 108)
I20260809 00:42:35.626421 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.626248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33195 (local address 127.0.0.1:35332)
0809 00:42:35.626322 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.626376 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:33195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 00:42:35.626592 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 5495
W20260809 00:42:35.626576 5203 catalog_manager.cc:4584] TS 3194166c3f0d40c8ab9732c8bb1ed124 (127.30.104.193:33195): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a39c6ec2bc646a2b344279bbd6184a1: Network error: Client connection negotiation failed: client connection to 127.30.104.193:33195: connect: Connection refused (error 111)
I20260809 00:42:35.637172 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 5671
I20260809 00:42:35.637308 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.637147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:35755 (local address 127.0.0.1:39922)
0809 00:42:35.637229 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.637260 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.194:35755: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 00:42:35.650053 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 5176
2026-08-09T00:42:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (725 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 00:42:35.675073 31139 mini_oidc.cc:152] Starting JWKS server
I20260809 00:42:35.676307 31139 webserver.cc:460] Webserver started at https://127.0.0.1:46567/ using document root <none> and password file <none>
I20260809 00:42:35.676450 31139 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:42:35.676659 31139 webserver.cc:460] Webserver started at http://127.0.0.1:40351/ using document root <none> and password file <none>
2026-08-09T00:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:35Z Disabled control of system clock
I20260809 00:42:35.686640 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:38061
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:37371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:38061
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46567/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/testchainca.cert with env {}
W20260809 00:42:35.775677 5908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:35.775909 5908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:35.775934 5908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:35.777212 5908 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:35.777259 5908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:35.777273 5908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:35.777325 5908 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:35.777347 5908 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:42:35.777369 5908 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:42:35.777400 5908 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46567/jwks.json
I20260809 00:42:35.779047 5908 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:38061
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:38061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46567/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:35.779289 5908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:35.779505 5908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:35.784687 5917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:35.784750 5919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:35.784968 5908 server_base.cc:1034] running on GCE node
W20260809 00:42:35.784691 5916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:35.785341 5908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:35.785673 5908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:35.786851 5908 hybrid_clock.cc:648] HybridClock initialized: now 1786236155786826 us; error 55 us; skew 500 ppm
I20260809 00:42:35.788336 5908 webserver.cc:460] Webserver started at http://127.30.104.254:40283/ using document root <none> and password file <none>
I20260809 00:42:35.788625 5908 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:35.788704 5908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:35.788864 5908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:35.790024 5908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "07b9a8eb44464bb79c1926e44be79e32"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "0804690e97ad4353f9eedd9e161a7251"
server_key_iv: "ca0b2bad907a9ff1eb9d55fbfa464ca9"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.790423 5908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "07b9a8eb44464bb79c1926e44be79e32"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "0804690e97ad4353f9eedd9e161a7251"
server_key_iv: "ca0b2bad907a9ff1eb9d55fbfa464ca9"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.791898 5908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:35.792706 5929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:35.792914 5908 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:35.793047 5908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "07b9a8eb44464bb79c1926e44be79e32"
format_stamp: "Formatted at 2026-08-09 00:42:35 on dist-test-slave-c12x"
server_key: "0804690e97ad4353f9eedd9e161a7251"
server_key_iv: "ca0b2bad907a9ff1eb9d55fbfa464ca9"
server_key_version: "encryptionkey@0"
I20260809 00:42:35.793131 5908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:35.817150 5908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:35.817549 5908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:35.817730 5908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:35.822569 5908 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:38061
I20260809 00:42:35.822773 6004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:38061 every 8 connection(s)
I20260809 00:42:35.823050 5908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:35.824196 6006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:35.826607 6006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32: Bootstrap starting.
I20260809 00:42:35.827245 6006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:35.827607 6006 log.cc:824] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:35.828341 6006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32: No bootstrap required, opened a new log
I20260809 00:42:35.830019 6006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b9a8eb44464bb79c1926e44be79e32" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38061 } }
I20260809 00:42:35.830168 6006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:35.830191 6006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b9a8eb44464bb79c1926e44be79e32, State: Initialized, Role: FOLLOWER
I20260809 00:42:35.830348 6006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b9a8eb44464bb79c1926e44be79e32" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38061 } }
I20260809 00:42:35.830435 6006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:35.830457 6006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:35.830524 6006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:35.831064 6006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b9a8eb44464bb79c1926e44be79e32" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38061 } }
I20260809 00:42:35.831156 6006 leader_election.cc:302] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07b9a8eb44464bb79c1926e44be79e32; no voters:
I20260809 00:42:35.831364 6006 leader_election.cc:288] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:35.831545 6009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:35.831712 6006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:35.831712 6009 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [term 1 LEADER]: Becoming Leader. State: Replica: 07b9a8eb44464bb79c1926e44be79e32, State: Running, Role: LEADER
I20260809 00:42:35.831879 6009 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b9a8eb44464bb79c1926e44be79e32" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38061 } }
I20260809 00:42:35.832346 6011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07b9a8eb44464bb79c1926e44be79e32. Latest consensus state: current_term: 1 leader_uuid: "07b9a8eb44464bb79c1926e44be79e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b9a8eb44464bb79c1926e44be79e32" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38061 } } }
I20260809 00:42:35.832432 6010 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07b9a8eb44464bb79c1926e44be79e32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07b9a8eb44464bb79c1926e44be79e32" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38061 } } }
I20260809 00:42:35.832461 6011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:35.832518 6010 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:35.832711 6024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:35.833036 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 5908
I20260809 00:42:35.833150 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:35.833277 6024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:35.833361 31139 external_mini_cluster.cc:1428] Setting key 222e4324bd876979d3c4f7b43c30587b
I20260809 00:42:35.834981 6024 catalog_manager.cc:1347] Generated new cluster ID: 00465e986fd340e8a6ad7959b8c0e192
I20260809 00:42:35.835039 6024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:35.839850 6037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.834031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.30.104.254:38061)
0809 00:42:35.834428 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.834438 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.834476 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:42:35.834603 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.834609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.834663 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.834807 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:35.835039 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:35.836439 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:35.837547 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:35.838240 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:35.838373 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:35.839069 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:35.839073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:35.839074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:35.839300 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:35.839344 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:35.839358 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:35.839453 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:35.839518 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 00:42:35.839600 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":75,"threads_started":1}
I20260809 00:42:35.841180 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:42:35.856879 6024 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:35.857545 6024 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:35.873838 6024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07b9a8eb44464bb79c1926e44be79e32: Generated new TSK 0
I20260809 00:42:35.874101 6024 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 00:42:35.905973 6048 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 00:42:35.906121 6048 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 00:42:35.906359 6037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:35.889733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.30.104.254:38061)
0809 00:42:35.889811 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:35.889813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:35.890033 (+ 220us) server_negotiation.cc:407] Connection header received
0809 00:42:35.890149 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:35.890150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:35.890164 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:35.890245 (+ 81us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:42:35.890419 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:35.891216 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:35.892149 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:35.892444 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:35.892525 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:35.892874 (+ 349us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:42:35.906099 (+ 13225us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 00:42:35.906280 (+ 181us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32838: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 00:42:35.906433 6037 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32838: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 00:42:35.906966 6047 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:35.889601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:38061 (local address 127.0.0.1:32838)
0809 00:42:35.889947 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:35.889959 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:35.890047 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:35.890269 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:35.890269 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:35.890271 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:42:35.890345 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:35.890347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:35.891333 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:35.891334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:35.892057 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:35.892059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:35.892575 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:35.892576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:35.892708 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:35.892736 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:42:35.906369 (+ 13633us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 00:42:35.906649 (+ 280us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.104.254:38061: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":159,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.104.254:38061: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46567/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 00:42:35.907557 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 5908
2026-08-09T00:42:35Z chronyd exiting
I20260809 00:42:35.957736 31139 test_util.cc:175] -----------------------------------------------
I20260809 00:42:35.957858 31139 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786236131192310-31139-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (283 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 00:42:35.958662 31139 mini_oidc.cc:152] Starting JWKS server
I20260809 00:42:35.959378 31139 webserver.cc:460] Webserver started at https://127.0.0.1:45325/ using document root <none> and password file <none>
I20260809 00:42:35.959481 31139 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:42:35.959692 31139 webserver.cc:460] Webserver started at http://127.0.0.1:34501/ using document root <none> and password file <none>
2026-08-09T00:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:35Z Disabled control of system clock
I20260809 00:42:35.968940 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37397
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:38673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37397
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45325/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/testchainca.cert with env {}
W20260809 00:42:36.053093 6055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:36.053319 6055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:36.053344 6055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:36.054581 6055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:36.054628 6055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:36.054641 6055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:36.054653 6055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:36.054665 6055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:42:36.054710 6055 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:42:36.054739 6055 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45325/jwks.json
I20260809 00:42:36.056360 6055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37397
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45325/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:36.056661 6055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:36.056886 6055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:36.062330 6063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:36.062301 6055 server_base.cc:1034] running on GCE node
W20260809 00:42:36.062315 6061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:36.062301 6060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:36.062850 6055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:36.063201 6055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:36.064378 6055 hybrid_clock.cc:648] HybridClock initialized: now 1786236156064348 us; error 52 us; skew 500 ppm
I20260809 00:42:36.065734 6055 webserver.cc:460] Webserver started at http://127.30.104.254:38435/ using document root <none> and password file <none>
I20260809 00:42:36.066025 6055 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:36.066077 6055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:36.066229 6055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:36.067361 6055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "4602d143ee634002ab6c17132c1e1c25"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "db783c646cd75ababc5e54081b6f79df"
server_key_iv: "c6cc1ac49cb046f824051f85754b7738"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.067827 6055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "4602d143ee634002ab6c17132c1e1c25"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "db783c646cd75ababc5e54081b6f79df"
server_key_iv: "c6cc1ac49cb046f824051f85754b7738"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.069284 6055 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:36.070147 6071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:36.070353 6055 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:36.070473 6055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "4602d143ee634002ab6c17132c1e1c25"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "db783c646cd75ababc5e54081b6f79df"
server_key_iv: "c6cc1ac49cb046f824051f85754b7738"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.070556 6055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:36.084347 6055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:36.084710 6055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:36.084872 6055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:36.089088 6055 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37397
I20260809 00:42:36.089147 6144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37397 every 8 connection(s)
I20260809 00:42:36.089531 6055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:36.090294 6145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:36.093056 6145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25: Bootstrap starting.
I20260809 00:42:36.093636 6145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:36.093885 6145 log.cc:824] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:36.094224 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 6055
I20260809 00:42:36.094347 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:36.094537 31139 external_mini_cluster.cc:1428] Setting key f152164e46fd709096747e22314553f5
I20260809 00:42:36.094801 6145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25: No bootstrap required, opened a new log
I20260809 00:42:36.096715 6145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602d143ee634002ab6c17132c1e1c25" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37397 } }
I20260809 00:42:36.096902 6145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:36.096952 6145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4602d143ee634002ab6c17132c1e1c25, State: Initialized, Role: FOLLOWER
I20260809 00:42:36.097091 6145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602d143ee634002ab6c17132c1e1c25" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37397 } }
I20260809 00:42:36.097193 6145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:36.097234 6145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:36.097289 6145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:36.097859 6145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602d143ee634002ab6c17132c1e1c25" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37397 } }
I20260809 00:42:36.097982 6145 leader_election.cc:302] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4602d143ee634002ab6c17132c1e1c25; no voters:
I20260809 00:42:36.098152 6145 leader_election.cc:288] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:36.098206 6152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:36.098366 6152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [term 1 LEADER]: Becoming Leader. State: Replica: 4602d143ee634002ab6c17132c1e1c25, State: Running, Role: LEADER
I20260809 00:42:36.098430 6145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:36.098484 6152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602d143ee634002ab6c17132c1e1c25" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37397 } }
I20260809 00:42:36.098819 6152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4602d143ee634002ab6c17132c1e1c25. Latest consensus state: current_term: 1 leader_uuid: "4602d143ee634002ab6c17132c1e1c25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602d143ee634002ab6c17132c1e1c25" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37397 } } }
I20260809 00:42:36.098881 6152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:36.098971 6155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4602d143ee634002ab6c17132c1e1c25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602d143ee634002ab6c17132c1e1c25" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37397 } } }
I20260809 00:42:36.099131 6155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:36.099781 6159 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:36.100448 6159 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:36.102566 6159 catalog_manager.cc:1347] Generated new cluster ID: 80447405d7d444e4a2ddaaceb040dfec
I20260809 00:42:36.102618 6159 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:36.104107 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:36.095070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56938 (local address 127.30.104.254:37397)
0809 00:42:36.095342 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:36.095352 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:36.095390 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:42:36.096710 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:36.096719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:36.096794 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:36.096983 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:36.097241 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.099077 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.100445 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.101123 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.101273 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.102136 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:36.102140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:36.102141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:36.102349 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:36.102387 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:36.102400 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:36.103561 (+ 1161us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:36.103656 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 00:42:36.103863 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":95,"threads_started":1}
I20260809 00:42:36.105110 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:42:36.110525 6159 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:36.111133 6159 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:36.123095 6159 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4602d143ee634002ab6c17132c1e1c25: Generated new TSK 0
I20260809 00:42:36.123339 6159 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 00:42:36.152195 6184 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 00:42:36.152297 6184 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 00:42:36.152608 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:36.129731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56950 (local address 127.30.104.254:37397)
0809 00:42:36.129781 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:36.129784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:36.129874 (+ 90us) server_negotiation.cc:407] Connection header received
0809 00:42:36.129964 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:36.129965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:36.129978 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:36.130076 (+ 98us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:42:36.130340 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.130996 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.131946 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.132193 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.132287 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.132494 (+ 207us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:42:36.152358 (+ 19864us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 00:42:36.152549 (+ 191us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:42:36.152691 6151 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56950: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 00:42:36.152729 6183 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:36.129644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37397 (local address 127.0.0.1:56950)
0809 00:42:36.129810 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:36.129820 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:36.129883 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:36.130076 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:36.130077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:36.130079 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:42:36.130142 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:36.130145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.131127 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.131128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:36.131854 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:36.131858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.132295 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.132296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:36.132362 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.132367 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:42:36.152508 (+ 20141us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 00:42:36.152630 (+ 122us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.104.254:37397: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260809 00:42:36.153076 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 6055
2026-08-09T00:42:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (240 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 00:42:36.199075 31139 mini_oidc.cc:152] Starting JWKS server
I20260809 00:42:36.199891 31139 webserver.cc:460] Webserver started at https://127.0.0.1:37943/ using document root <none> and password file <none>
I20260809 00:42:36.199965 31139 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 00:42:36.200132 31139 webserver.cc:460] Webserver started at http://127.0.0.1:44965/ using document root <none> and password file <none>
2026-08-09T00:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:36Z Disabled control of system clock
I20260809 00:42:36.209689 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33537
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:33537
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37943/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 00:42:36.295894 6194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:36.296129 6194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:36.296152 6194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:36.297291 6194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:36.297330 6194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:36.297343 6194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:36.297355 6194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:36.297367 6194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:42:36.297380 6194 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:42:36.297391 6194 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37943/jwks.json
I20260809 00:42:36.298985 6194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:33537
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37943/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:36.299216 6194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:36.299480 6194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:36.304646 6202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:36.304657 6205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:36.304714 6201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:36.304904 6194 server_base.cc:1034] running on GCE node
I20260809 00:42:36.305239 6194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:36.305540 6194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:36.306697 6194 hybrid_clock.cc:648] HybridClock initialized: now 1786236156306672 us; error 46 us; skew 500 ppm
I20260809 00:42:36.308054 6194 webserver.cc:460] Webserver started at http://127.30.104.254:35005/ using document root <none> and password file <none>
I20260809 00:42:36.308303 6194 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:36.308372 6194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:36.308542 6194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:36.309695 6194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "b7d2b18b2cbc4f12adc47a739abcad50"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "638b725f3e3bb4824082c471cd8f58a9"
server_key_iv: "d9b7e32dddbedbd920cff67f93999a68"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.310076 6194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "b7d2b18b2cbc4f12adc47a739abcad50"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "638b725f3e3bb4824082c471cd8f58a9"
server_key_iv: "d9b7e32dddbedbd920cff67f93999a68"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.311551 6194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:42:36.312461 6215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:36.312654 6194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:36.312778 6194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "b7d2b18b2cbc4f12adc47a739abcad50"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "638b725f3e3bb4824082c471cd8f58a9"
server_key_iv: "d9b7e32dddbedbd920cff67f93999a68"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.312860 6194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:36.324386 6194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:36.324723 6194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:36.324877 6194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:36.329156 6194 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:33537
I20260809 00:42:36.329188 6284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:33537 every 8 connection(s)
I20260809 00:42:36.329572 6194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:36.330292 6286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:36.333107 6286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50: Bootstrap starting.
I20260809 00:42:36.333660 6286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:36.333901 6286 log.cc:824] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:36.334578 6286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50: No bootstrap required, opened a new log
I20260809 00:42:36.334884 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 6194
I20260809 00:42:36.334990 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:36.335155 31139 external_mini_cluster.cc:1428] Setting key 49a1587514119ea86aa8ee5be7a57283
I20260809 00:42:36.337182 6286 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d2b18b2cbc4f12adc47a739abcad50" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33537 } }
I20260809 00:42:36.337353 6286 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:36.337392 6286 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7d2b18b2cbc4f12adc47a739abcad50, State: Initialized, Role: FOLLOWER
I20260809 00:42:36.337532 6286 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d2b18b2cbc4f12adc47a739abcad50" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33537 } }
I20260809 00:42:36.337632 6286 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:36.337669 6286 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:36.337719 6286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:36.338279 6286 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d2b18b2cbc4f12adc47a739abcad50" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33537 } }
I20260809 00:42:36.338370 6286 leader_election.cc:302] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7d2b18b2cbc4f12adc47a739abcad50; no voters:
I20260809 00:42:36.338584 6286 leader_election.cc:288] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:36.338624 6291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:36.338768 6291 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [term 1 LEADER]: Becoming Leader. State: Replica: b7d2b18b2cbc4f12adc47a739abcad50, State: Running, Role: LEADER
I20260809 00:42:36.338855 6291 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d2b18b2cbc4f12adc47a739abcad50" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33537 } }
I20260809 00:42:36.338894 6286 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:36.339145 6291 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7d2b18b2cbc4f12adc47a739abcad50. Latest consensus state: current_term: 1 leader_uuid: "b7d2b18b2cbc4f12adc47a739abcad50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d2b18b2cbc4f12adc47a739abcad50" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33537 } } }
I20260809 00:42:36.339210 6291 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:36.339385 6292 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7d2b18b2cbc4f12adc47a739abcad50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7d2b18b2cbc4f12adc47a739abcad50" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33537 } } }
I20260809 00:42:36.339493 6292 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:36.339628 6300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:36.340291 6300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:36.342470 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:36.335752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44974 (local address 127.30.104.254:33537)
0809 00:42:36.336047 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:36.336056 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:36.336126 (+ 70us) server_negotiation.cc:407] Connection header received
0809 00:42:36.337163 (+ 1037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:36.337170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:36.337234 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:36.337380 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:36.337663 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.339256 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.340412 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.340955 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.341051 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.341823 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:36.341827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:36.341827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:36.341997 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:36.342031 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:36.342041 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:36.342120 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:36.342192 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:42:36.342275 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
I20260809 00:42:36.343487 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:42:36.343947 6300 catalog_manager.cc:1347] Generated new cluster ID: b595d69db1f04bc0baf999d8aef5c542
I20260809 00:42:36.343991 6300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:36.356468 6300 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:36.357010 6300 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:36.363261 6300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7d2b18b2cbc4f12adc47a739abcad50: Generated new TSK 0
I20260809 00:42:36.363521 6300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:42:36.380437 6290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:36.376503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.30.104.254:33537)
0809 00:42:36.376555 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:36.376557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:36.376599 (+ 42us) server_negotiation.cc:407] Connection header received
0809 00:42:36.376694 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:36.376695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:36.376704 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:36.376768 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:42:36.376934 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.377423 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.378600 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.378794 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.378851 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.379094 (+ 243us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:42:36.380185 (+ 1091us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37943/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 00:42:36.380328 (+ 143us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37943/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:42:36.380527 6290 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44976: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37943/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 00:42:36.380543 6328 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:36.376354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33537 (local address 127.0.0.1:44976)
0809 00:42:36.376535 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:36.376547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:36.376620 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:36.376793 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:36.376794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:36.376796 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:42:36.376859 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:36.376862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.377516 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.377517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:36.378502 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:36.378505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.378886 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.378887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:36.378953 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.378959 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:42:36.380307 (+ 1348us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37943/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 00:42:36.380427 (+ 120us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.104.254:33537: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37943/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260809 00:42:36.380724 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 6194
E20260809 00:42:36.400266 6330 webserver.cc:533] Webserver: sslize failed:
E20260809 00:42:36.400358 6330 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T00:42:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (231 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T00:42:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:36Z Disabled control of system clock
I20260809 00:42:36.440704 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37245
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:41305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37245
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/testchainca.cert with env {}
W20260809 00:42:36.526214 6334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:36.526491 6334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:36.526626 6334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:36.528112 6334 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:36.528190 6334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:36.528213 6334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:36.528242 6334 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:36.528291 6334 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 00:42:36.528323 6334 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 00:42:36.528353 6334 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 00:42:36.530917 6334 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37245
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:36.531258 6334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:36.531567 6334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:36.537041 6340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:36.537042 6339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:36.537124 6334 server_base.cc:1034] running on GCE node
W20260809 00:42:36.537046 6344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:36.537611 6334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:36.537943 6334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:36.539122 6334 hybrid_clock.cc:648] HybridClock initialized: now 1786236156539102 us; error 33 us; skew 500 ppm
I20260809 00:42:36.540597 6334 webserver.cc:460] Webserver started at http://127.30.104.254:33485/ using document root <none> and password file <none>
I20260809 00:42:36.540815 6334 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:36.540859 6334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:36.540954 6334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:36.542053 6334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "a66aa65e4ac8481684a4e8f0aee6e528"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "3c798c9aa7988cbe4e0a0fc570062989"
server_key_iv: "fe8f4cd1551adaecf851f084ec4e5e0c"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.542382 6334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "a66aa65e4ac8481684a4e8f0aee6e528"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "3c798c9aa7988cbe4e0a0fc570062989"
server_key_iv: "fe8f4cd1551adaecf851f084ec4e5e0c"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.543843 6334 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:36.544687 6353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:36.544883 6334 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:36.544989 6334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a66aa65e4ac8481684a4e8f0aee6e528"
format_stamp: "Formatted at 2026-08-09 00:42:36 on dist-test-slave-c12x"
server_key: "3c798c9aa7988cbe4e0a0fc570062989"
server_key_iv: "fe8f4cd1551adaecf851f084ec4e5e0c"
server_key_version: "encryptionkey@0"
I20260809 00:42:36.545056 6334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:36.575141 6334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:36.575522 6334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:36.575647 6334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:36.579478 6334 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37245
I20260809 00:42:36.579507 6424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37245 every 8 connection(s)
I20260809 00:42:36.579902 6334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:36.581004 6425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:36.583416 6425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528: Bootstrap starting.
I20260809 00:42:36.584103 6425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:36.584410 6425 log.cc:824] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:36.585189 6425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528: No bootstrap required, opened a new log
I20260809 00:42:36.586792 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 6334
I20260809 00:42:36.586889 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:36.586926 6425 raft_consensus.cc:348] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66aa65e4ac8481684a4e8f0aee6e528" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37245 } }
I20260809 00:42:36.587085 31139 external_mini_cluster.cc:1428] Setting key 1653a6b08db2a694642025ef5a2c03a3
I20260809 00:42:36.587119 6425 raft_consensus.cc:374] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:36.587172 6425 raft_consensus.cc:729] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a66aa65e4ac8481684a4e8f0aee6e528, State: Initialized, Role: FOLLOWER
I20260809 00:42:36.587355 6425 consensus_queue.cc:260] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66aa65e4ac8481684a4e8f0aee6e528" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37245 } }
I20260809 00:42:36.587491 6425 raft_consensus.cc:388] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:36.587533 6425 raft_consensus.cc:482] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:36.587592 6425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:36.588335 6425 raft_consensus.cc:504] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66aa65e4ac8481684a4e8f0aee6e528" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37245 } }
I20260809 00:42:36.588459 6425 leader_election.cc:302] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a66aa65e4ac8481684a4e8f0aee6e528; no voters:
I20260809 00:42:36.588661 6425 leader_election.cc:288] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:36.588829 6435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:36.588959 6435 raft_consensus.cc:686] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [term 1 LEADER]: Becoming Leader. State: Replica: a66aa65e4ac8481684a4e8f0aee6e528, State: Running, Role: LEADER
I20260809 00:42:36.589000 6425 sys_catalog.cc:564] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:36.589049 6435 consensus_queue.cc:237] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66aa65e4ac8481684a4e8f0aee6e528" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37245 } }
I20260809 00:42:36.589385 6435 sys_catalog.cc:455] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a66aa65e4ac8481684a4e8f0aee6e528. Latest consensus state: current_term: 1 leader_uuid: "a66aa65e4ac8481684a4e8f0aee6e528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66aa65e4ac8481684a4e8f0aee6e528" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37245 } } }
I20260809 00:42:36.589505 6435 sys_catalog.cc:458] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:36.589398 6436 sys_catalog.cc:455] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a66aa65e4ac8481684a4e8f0aee6e528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a66aa65e4ac8481684a4e8f0aee6e528" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37245 } } }
I20260809 00:42:36.589644 6436 sys_catalog.cc:458] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:36.590154 6442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:36.590679 6442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:36.592527 6442 catalog_manager.cc:1347] Generated new cluster ID: bbec4e517f6d46ad9d97a77881476503
I20260809 00:42:36.592592 6442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:36.593122 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:36.587648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55402 (local address 127.30.104.254:37245)
0809 00:42:36.587862 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:36.587870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:36.587895 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:42:36.588000 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:36.588012 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:36.588085 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:36.588194 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:36.588485 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.589828 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.590919 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.591538 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.591674 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.592376 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:36.592380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:36.592381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:42:36.592609 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:36.592653 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:42:36.592669 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:36.592739 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:36.592845 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 00:42:36.592919 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260809 00:42:36.596612 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:42:36.606186 6442 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:36.606787 6442 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:36.612958 6442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a66aa65e4ac8481684a4e8f0aee6e528: Generated new TSK 0
I20260809 00:42:36.613189 6442 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:42:36.674989 6433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:36.624543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.30.104.254:37245)
0809 00:42:36.624611 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:36.624613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:36.624724 (+ 111us) server_negotiation.cc:407] Connection header received
0809 00:42:36.624784 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:36.624786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:36.624800 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:36.624897 (+ 97us) server_negotiation.cc:226] Negotiated authn=JWT
0809 00:42:36.625138 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.625954 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.626789 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.627131 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.627204 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.627465 (+ 261us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 00:42:36.674529 (+ 47064us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 00:42:36.674895 (+ 366us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55404: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 00:42:36.675073 6467 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:36.624418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37245 (local address 127.0.0.1:55404)
0809 00:42:36.624633 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:36.624644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:36.624722 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:36.624982 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:36.624983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:36.624986 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0809 00:42:36.625049 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:36.625051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.626075 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.626076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:36.626698 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:36.626700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:36.627253 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:36.627254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:36.627309 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:36.627315 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 00:42:36.674813 (+ 47498us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 00:42:36.674957 (+ 144us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.104.254:37245: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":145,"threads_started":1}
W20260809 00:42:36.675103 6433 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55404: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260809 00:42:36.675422 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 6334
2026-08-09T00:42:36Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (273 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (95 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:36 dist-test-slave-c12x krb5kdc[6490](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:36 dist-test-slave-c12x krb5kdc[6490](info): set up 2 sockets
Aug 09 00:42:36 dist-test-slave-c12x krb5kdc[6490](info): commencing operation
krb5kdc: starting...
W20260809 00:42:38.309892 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.415s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 00:42:38.559692 31139 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.250s user 0.000s sys 0.003s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 00:42:38.761171 31139 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.201s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 00:42:38.926367 31139 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.165s user 0.001s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/test-user.keytab.
W20260809 00:42:39.078918 31139 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.152s user 0.003s sys 0.000s
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:39.131289 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:46001
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:37427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:46001
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:39.218835 6507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:39.219076 6507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:39.219102 6507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:39.220417 6507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:39.220463 6507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:39.220476 6507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:39.220490 6507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:39.220502 6507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:39.222059 6507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:46001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:46001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:39.222339 6507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:39.222591 6507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:39.228129 6513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:39.228134 6514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:39.228307 6517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:39.228448 6507 server_base.cc:1034] running on GCE node
I20260809 00:42:39.228737 6507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:39.229071 6507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:39.230270 6507 hybrid_clock.cc:648] HybridClock initialized: now 1786236159230273 us; error 50 us; skew 500 ppm
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.236267 6507 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:39.236866 6507 webserver.cc:460] Webserver started at http://127.30.104.254:46627/ using document root <none> and password file <none>
I20260809 00:42:39.237125 6507 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:39.237207 6507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:39.237375 6507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:39.238906 6507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "9d0bd7da1440462a984c84e08c48a357"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "0b8e80a0a8e32b9cdacf46bfa76380b5"
server_key_iv: "cb43739612889502e296f8ef3b8cdae8"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.239459 6507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "9d0bd7da1440462a984c84e08c48a357"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "0b8e80a0a8e32b9cdacf46bfa76380b5"
server_key_iv: "cb43739612889502e296f8ef3b8cdae8"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.241377 6507 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:39.242304 6526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.242555 6507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:39.242671 6507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "9d0bd7da1440462a984c84e08c48a357"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "0b8e80a0a8e32b9cdacf46bfa76380b5"
server_key_iv: "cb43739612889502e296f8ef3b8cdae8"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.242765 6507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:39.255081 6507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:39.257421 6507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:39.257612 6507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:39.261883 6507 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:46001
I20260809 00:42:39.261943 6609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:46001 every 8 connection(s)
I20260809 00:42:39.262341 6507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:39.263372 6611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:39.266005 6611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357: Bootstrap starting.
I20260809 00:42:39.266708 6611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:39.267051 6611 log.cc:824] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:39.267760 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 6507
I20260809 00:42:39.267911 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:39.268139 31139 external_mini_cluster.cc:1428] Setting key 21a4aa8a82c901b6f0e56c958d49aa9f
I20260809 00:42:39.268241 6611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357: No bootstrap required, opened a new log
I20260809 00:42:39.271152 6611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0bd7da1440462a984c84e08c48a357" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46001 } }
I20260809 00:42:39.271394 6611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:39.271492 6611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d0bd7da1440462a984c84e08c48a357, State: Initialized, Role: FOLLOWER
I20260809 00:42:39.271687 6611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0bd7da1440462a984c84e08c48a357" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46001 } }
I20260809 00:42:39.271785 6611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:39.271824 6611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:39.271868 6611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:39.272718 6611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0bd7da1440462a984c84e08c48a357" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46001 } }
I20260809 00:42:39.272843 6611 leader_election.cc:302] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9d0bd7da1440462a984c84e08c48a357; no voters:
I20260809 00:42:39.273028 6611 leader_election.cc:288] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:39.273095 6619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:39.273315 6619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [term 1 LEADER]: Becoming Leader. State: Replica: 9d0bd7da1440462a984c84e08c48a357, State: Running, Role: LEADER
I20260809 00:42:39.273358 6611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:39.273418 6619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0bd7da1440462a984c84e08c48a357" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46001 } }
I20260809 00:42:39.273844 6619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d0bd7da1440462a984c84e08c48a357. Latest consensus state: current_term: 1 leader_uuid: "9d0bd7da1440462a984c84e08c48a357" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0bd7da1440462a984c84e08c48a357" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46001 } } }
I20260809 00:42:39.273926 6620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9d0bd7da1440462a984c84e08c48a357" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9d0bd7da1440462a984c84e08c48a357" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46001 } } }
I20260809 00:42:39.273965 6619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:39.274036 6620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:39.274323 6631 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:39.275563 6631 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:39.277549 6631 catalog_manager.cc:1347] Generated new cluster ID: bf12167653e24993893a372f32359fee
I20260809 00:42:39.277611 6631 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:39.282685 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.268733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.30.104.254:46001)
0809 00:42:39.268972 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.268984 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.269028 (+ 44us) server_negotiation.cc:407] Connection header received
0809 00:42:39.269991 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.270002 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.270075 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.270253 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.270949 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.272909 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.274120 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.275003 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.275162 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.278324 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:39.278328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:39.278329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:39.278343 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:39.280593 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.281129 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.281133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.281134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.281261 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.281493 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.281494 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.281494 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.281844 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:39.282011 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:39.282240 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 00:42:39.282413 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":125,"threads_started":1}
I20260809 00:42:39.286051 6631 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:39.286794 6631 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:39.297075 6631 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d0bd7da1440462a984c84e08c48a357: Generated new TSK 0
I20260809 00:42:39.297377 6631 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:39.327909 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:46001
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:37427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:39.416630 6657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:39.416873 6657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:39.416900 6657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:39.418154 6657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:39.418210 6657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:39.418241 6657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:39.419945 6657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:39.420256 6657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:39.420512 6657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:39.426403 6667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:39.426441 6665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:39.426441 6670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:39.427161 6657 server_base.cc:1034] running on GCE node
I20260809 00:42:39.427513 6657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:39.427906 6657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:39.429088 6657 hybrid_clock.cc:648] HybridClock initialized: now 1786236159429036 us; error 71 us; skew 500 ppm
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.432220 6657 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:39.432719 6657 webserver.cc:460] Webserver started at http://127.30.104.193:43153/ using document root <none> and password file <none>
I20260809 00:42:39.433002 6657 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:39.433175 6657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:39.433360 6657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:39.434634 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "4b23d5209c89405dad07d6b9abf1c038"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "7a96b7415d9ab19f8f797048a90f1462"
server_key_iv: "315c8787d3ca062c4588236e331ebb7b"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.435046 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "4b23d5209c89405dad07d6b9abf1c038"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "7a96b7415d9ab19f8f797048a90f1462"
server_key_iv: "315c8787d3ca062c4588236e331ebb7b"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.436928 6657 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 00:42:39.438221 6680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.438581 6657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:42:39.438726 6657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "4b23d5209c89405dad07d6b9abf1c038"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "7a96b7415d9ab19f8f797048a90f1462"
server_key_iv: "315c8787d3ca062c4588236e331ebb7b"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.438822 6657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:39.456139 6657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:39.459138 6657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:39.459328 6657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:39.459652 6657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:39.460011 6657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:39.460069 6657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.460114 6657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:39.460139 6657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.466367 6657 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:45267
I20260809 00:42:39.466413 6847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:45267 every 8 connection(s)
I20260809 00:42:39.466764 6657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:39.475464 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 6657
I20260809 00:42:39.475590 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:39.475816 31139 external_mini_cluster.cc:1428] Setting key 50bc9d6b77b09bb5a5535a6283253e48
I20260809 00:42:39.478194 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.467701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:49927 (local address 127.30.104.254:46001)
0809 00:42:39.467790 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.467793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.468039 (+ 246us) server_negotiation.cc:407] Connection header received
0809 00:42:39.469184 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.469185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.469196 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.469277 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.470038 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.470576 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.471804 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.472071 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.472146 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.474630 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:39.474632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:39.474633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:39.474642 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:39.476528 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.477093 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.477094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.477095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.477168 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.477472 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.477473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.477473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.477603 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:39.477678 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:39.477997 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 00:42:39.478097 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:42:39.478415 6851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.467644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:46001 (local address 127.30.104.193:49927)
0809 00:42:39.467868 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:39.467913 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:39.469036 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:39.469377 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:39.469381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:39.469702 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:39.469930 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:39.469935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.470674 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.470675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:39.471695 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:39.471698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.472165 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.472166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:39.472290 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.472903 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:39.472912 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:39.474485 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:39.476647 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:39.476657 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:39.476660 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:39.476990 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:39.477238 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:39.477238 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:39.477239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:39.477367 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:39.477682 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:39.477690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:39.477838 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 00:42:39.477970 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 00:42:39.478147 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":102,"threads_started":1}
I20260809 00:42:39.479004 6848 heartbeater.cc:347] Connected to a master server at 127.30.104.254:46001
I20260809 00:42:39.479110 6848 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:39.479343 6848 heartbeater.cc:511] Master 127.30.104.254:46001 requested a full tablet report, sending...
I20260809 00:42:39.480103 6553 ts_manager.cc:194] Registered new tserver with Master: 4b23d5209c89405dad07d6b9abf1c038 (127.30.104.193:45267)
I20260809 00:42:39.481048 6553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:49927
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:39.507491 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:46001
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:37427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:39.595615 6854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:39.595832 6854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:39.595856 6854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:39.597100 6854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:39.597148 6854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:39.597175 6854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:39.598690 6854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:39.598954 6854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:39.599205 6854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:42:39.605351 6854 server_base.cc:1034] running on GCE node
W20260809 00:42:39.605370 6864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:39.605351 6861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:39.605352 6859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:39.605850 6854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:39.606164 6854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:39.607394 6854 hybrid_clock.cc:648] HybridClock initialized: now 1786236159607365 us; error 46 us; skew 500 ppm
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.610365 6854 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:39.611167 6854 webserver.cc:460] Webserver started at http://127.30.104.194:40969/ using document root <none> and password file <none>
I20260809 00:42:39.611394 6854 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:39.611527 6854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:39.611663 6854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:39.612824 6854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "864126ffba714c94b8220e3ed934b3ac"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "977c99f6faaf1f1f1d1e799b6867bc7d"
server_key_iv: "bfdac51a070ea329dc93487bc8a6c62a"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.613236 6854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "864126ffba714c94b8220e3ed934b3ac"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "977c99f6faaf1f1f1d1e799b6867bc7d"
server_key_iv: "bfdac51a070ea329dc93487bc8a6c62a"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.614850 6854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:39.616151 6874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.616394 6854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:39.616518 6854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "864126ffba714c94b8220e3ed934b3ac"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "977c99f6faaf1f1f1d1e799b6867bc7d"
server_key_iv: "bfdac51a070ea329dc93487bc8a6c62a"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.616609 6854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:39.630538 6854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:39.633092 6854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:39.633277 6854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:39.633566 6854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:39.633925 6854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:39.633994 6854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.634033 6854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:39.634071 6854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.640715 6854 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:43123
I20260809 00:42:39.640761 7023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:43123 every 8 connection(s)
I20260809 00:42:39.641131 6854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:39.643654 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 6854
I20260809 00:42:39.643812 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:39.643992 31139 external_mini_cluster.cc:1428] Setting key bd56b3dcd0853535373453b1424d9657
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:39.654011 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.642179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:43687 (local address 127.30.104.254:46001)
0809 00:42:39.642259 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.642262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.642521 (+ 259us) server_negotiation.cc:407] Connection header received
0809 00:42:39.643801 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.643803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.643817 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.643893 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.644836 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.645673 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.647024 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.647249 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.647338 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.650489 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:39.650490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:39.650491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:39.650499 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:39.652235 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.652801 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.652802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.652802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.652866 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.653204 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.653205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.653205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.653353 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:39.653422 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:39.653837 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 00:42:39.653934 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:42:39.654253 7026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.642134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:46001 (local address 127.30.104.194:43687)
0809 00:42:39.642341 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:39.642417 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:39.643668 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:39.644006 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:39.644009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:39.644486 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:39.644709 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:39.644714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.645872 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.645873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:39.646890 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:39.646893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.647344 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.647345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:39.647566 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.648315 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:39.648325 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:39.650368 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:39.652328 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:39.652337 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:39.652341 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:39.652696 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:39.652950 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:39.652951 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:39.652951 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:39.653106 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:39.653429 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:39.653436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:39.653642 (+ 206us) client_negotiation.cc:769] Sending connection context
0809 00:42:39.653784 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 00:42:39.654001 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
I20260809 00:42:39.654846 7024 heartbeater.cc:347] Connected to a master server at 127.30.104.254:46001
I20260809 00:42:39.654942 7024 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:39.655093 7024 heartbeater.cc:511] Master 127.30.104.254:46001 requested a full tablet report, sending...
I20260809 00:42:39.655560 6553 ts_manager.cc:194] Registered new tserver with Master: 864126ffba714c94b8220e3ed934b3ac (127.30.104.194:43123)
I20260809 00:42:39.655915 6553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:43687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:39.675621 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:46001
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:37427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:39.762379 7029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:39.762605 7029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:39.762629 7029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:39.763823 7029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:39.763871 7029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:39.763898 7029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:39.765393 7029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:46001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:39.765612 7029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:39.765805 7029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:39.771201 7038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:39.771171 7036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:39.771191 7035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:39.771571 7029 server_base.cc:1034] running on GCE node
I20260809 00:42:39.771823 7029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:39.772113 7029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:39.773379 7029 hybrid_clock.cc:648] HybridClock initialized: now 1786236159773354 us; error 41 us; skew 500 ppm
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.776633 7029 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:39.777225 7029 webserver.cc:460] Webserver started at http://127.30.104.195:43327/ using document root <none> and password file <none>
I20260809 00:42:39.777536 7029 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:39.777619 7029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:39.777791 7029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:39.779009 7029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "20c61fe806024393af2a839de13811a0"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "b92276374b435124190f8103c93a9419"
server_key_iv: "3959a5066dec7924fb34c9f0fb4abfcf"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.779482 7029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "20c61fe806024393af2a839de13811a0"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "b92276374b435124190f8103c93a9419"
server_key_iv: "3959a5066dec7924fb34c9f0fb4abfcf"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.781054 7029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:39.781948 7047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.782159 7029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:39.782277 7029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "20c61fe806024393af2a839de13811a0"
format_stamp: "Formatted at 2026-08-09 00:42:39 on dist-test-slave-c12x"
server_key: "b92276374b435124190f8103c93a9419"
server_key_iv: "3959a5066dec7924fb34c9f0fb4abfcf"
server_key_version: "encryptionkey@0"
I20260809 00:42:39.782372 7029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:39.803961 7029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:39.806375 7029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:39.806577 7029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:39.806855 7029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:39.807224 7029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:39.807286 7029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.807337 7029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:39.807374 7029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:39.814132 7029 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:38401
I20260809 00:42:39.814179 7212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:38401 every 8 connection(s)
I20260809 00:42:39.814551 7029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:39.822021 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 7029
I20260809 00:42:39.822139 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:39.822314 31139 external_mini_cluster.cc:1428] Setting key 93085c1d61697b0e3325ab29e310be33
I20260809 00:42:39.825330 6618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.815366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:41241 (local address 127.30.104.254:46001)
0809 00:42:39.815416 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.815419 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.815676 (+ 257us) server_negotiation.cc:407] Connection header received
0809 00:42:39.816800 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.816802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.816816 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.816873 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.817614 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.818179 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.819289 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.819580 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.819663 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.822138 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:39.822139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:39.822140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:39.822151 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:39.823903 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.824323 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.824324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.824324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.824399 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.824702 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.824703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.824703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.824828 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:39.824915 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:39.825186 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 00:42:39.825261 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:39.825495 7217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.815335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:46001 (local address 127.30.104.195:41241)
0809 00:42:39.815539 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:39.815586 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:39.816663 (+ 1077us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:39.816948 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:39.816953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:39.817269 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:39.817482 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:39.817486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.818283 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.818284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:39.819184 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:39.819186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.819670 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.819672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:39.819822 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.820354 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:39.820363 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:39.822001 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:39.823973 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:39.823979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:39.823981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:39.824227 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:39.824506 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:39.824507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:39.824507 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:39.824620 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:39.824916 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:39.824921 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:39.825045 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 00:42:39.825150 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:42:39.825281 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":119,"threads_started":1}
I20260809 00:42:39.825975 7215 heartbeater.cc:347] Connected to a master server at 127.30.104.254:46001
I20260809 00:42:39.826067 7215 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:39.826220 7215 heartbeater.cc:511] Master 127.30.104.254:46001 requested a full tablet report, sending...
I20260809 00:42:39.826589 6553 ts_manager.cc:194] Registered new tserver with Master: 20c61fe806024393af2a839de13811a0 (127.30.104.195:38401)
I20260809 00:42:39.826958 6553 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:41241
I20260809 00:42:39.834796 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/test-admin.keytab.
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.852833 31139 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 00:42:39.854943 7223 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:39.853883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45332 (local address 127.30.104.193:45267)
0809 00:42:39.854121 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.854126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.854138 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:42:39.854168 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.854170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.854185 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.854266 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.854824 (+ 558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45332: BlockingRecv error: recv got EOF from 127.0.0.1:45332 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":143,"threads_started":1}
W20260809 00:42:39.855022 31139 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.856230 31139 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.857383 31139 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:39.866312 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.858084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45342 (local address 127.30.104.193:45267)
0809 00:42:39.858153 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.858156 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.858166 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:39.858211 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.858212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.858223 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.858277 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.858686 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.859672 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.860439 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.860795 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.860857 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.863167 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:39.863175 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:39.863175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:39.863184 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:39.864946 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.865335 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.865339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.865339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.865401 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.865603 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.865604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.865604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.865751 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:39.865884 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:39.866055 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:42:39.866177 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 00:42:39.868655 7223 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:39.868047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45358 (local address 127.30.104.193:45267)
0809 00:42:39.868115 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.868117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.868126 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:39.868202 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.868203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.868213 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.868285 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.868591 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45358: BlockingRecv error: recv got EOF from 127.0.0.1:45358 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 00:42:39.868803 31139 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.869979 31139 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.871105 31139 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:39.879215 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.871709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.30.104.193:45267)
0809 00:42:39.871784 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.871786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.871795 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:39.871855 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.871856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.871867 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.871919 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.872268 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.872850 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.873622 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.873868 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.873939 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.876481 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:39.876482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:39.876483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:39.876490 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:39.878042 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.878418 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.878418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.878419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.878472 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.878647 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.878647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.878647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.878744 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:39.878818 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:39.878987 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 00:42:39.879058 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 00:42:39.881050 7223 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:39.880431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45378 (local address 127.30.104.193:45267)
0809 00:42:39.880519 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.880523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.880554 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:42:39.880608 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.880609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.880617 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.880672 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.881014 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45378: BlockingRecv error: recv got EOF from 127.0.0.1:45378 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 00:42:39.881201 31139 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786236131192310-31139-0/krb5kdc/krb5cc' of type FILE
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.882393 31139 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:39.883412 31139 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 00:42:39 dist-test-slave-c12x krb5kdc[6490](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:42:39.891533 7223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:39.884120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45392 (local address 127.30.104.193:45267)
0809 00:42:39.884195 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:39.884197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:39.884254 (+ 57us) server_negotiation.cc:407] Connection header received
0809 00:42:39.884294 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:39.884295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:39.884304 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:39.884369 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:39.884783 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.885301 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.886032 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:39.886225 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:39.886304 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:39.888771 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:39.888772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:39.888773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:39.888783 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:39.890173 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.890622 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.890623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.890624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.890694 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:39.890932 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:39.890933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:39.890933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:39.891051 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:39.891166 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:39.891299 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 00:42:39.891391 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:42:39.892382 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 6657
I20260809 00:42:39.906081 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 6854
I20260809 00:42:39.920573 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 7029
I20260809 00:42:39.935534 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 6507
2026-08-09T00:42:39Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3069 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:40 dist-test-slave-c12x krb5kdc[7240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:40 dist-test-slave-c12x krb5kdc[7240](info): set up 2 sockets
Aug 09 00:42:40 dist-test-slave-c12x krb5kdc[7240](info): commencing operation
krb5kdc: starting...
W20260809 00:42:41.220728 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:41 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:41Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.254.keytab.
Entry for principal oryx/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.254.keytab.
I20260809 00:42:41.358563 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:40589
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.254.keytab
--principal=oryx/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:40589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:41.444234 7258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:41.444646 7258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:41.444722 7258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:41.445987 7258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:41.446069 7258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:41.446096 7258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:41.446128 7258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:41.446154 7258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:41.448141 7258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:40589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.254.keytab
--principal=oryx/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:40589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:41.448472 7258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:41.448730 7258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:41.454239 7266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:41.454372 7264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:41.454239 7263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:41.454829 7258 server_base.cc:1034] running on GCE node
I20260809 00:42:41.455070 7258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:41.455344 7258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:41.456525 7258 hybrid_clock.cc:648] HybridClock initialized: now 1786236161456507 us; error 35 us; skew 500 ppm
Aug 09 00:42:41 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:41.459379 7258 init.cc:377] Logged in from keytab as oryx/127.30.104.254@KRBTEST.COM (short username oryx)
I20260809 00:42:41.459970 7258 webserver.cc:460] Webserver started at http://127.30.104.254:44909/ using document root <none> and password file <none>
I20260809 00:42:41.460216 7258 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:41.460287 7258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:41.460436 7258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:41.461556 7258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "1f8a3704ece84f08a8e873f37aba8dcb"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "0d13cfe95b551f0ca64d8747bbf3ab39"
server_key_iv: "21ff91e549b7c07a945dc6b9fa9b6d93"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.461947 7258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "1f8a3704ece84f08a8e873f37aba8dcb"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "0d13cfe95b551f0ca64d8747bbf3ab39"
server_key_iv: "21ff91e549b7c07a945dc6b9fa9b6d93"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.463515 7258 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:41.464470 7281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:41.464702 7258 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:41.464814 7258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "1f8a3704ece84f08a8e873f37aba8dcb"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "0d13cfe95b551f0ca64d8747bbf3ab39"
server_key_iv: "21ff91e549b7c07a945dc6b9fa9b6d93"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.464907 7258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:41.478550 7258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:41.480914 7258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:41.481103 7258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:41.485417 7258 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:40589
I20260809 00:42:41.485461 7353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:40589 every 8 connection(s)
I20260809 00:42:41.485858 7258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:41.486970 7355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:41.489435 7355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: Bootstrap starting.
I20260809 00:42:41.490010 7355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:41.490255 7355 log.cc:824] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:41.490962 7355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: No bootstrap required, opened a new log
I20260809 00:42:41.492784 7355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } }
I20260809 00:42:41.492944 7355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:41.492969 7355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f8a3704ece84f08a8e873f37aba8dcb, State: Initialized, Role: FOLLOWER
I20260809 00:42:41.493074 7355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } }
I20260809 00:42:41.493175 7355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:41.493214 7355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:41.493268 7355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:41.494184 7355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } }
I20260809 00:42:41.494320 7355 leader_election.cc:302] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f8a3704ece84f08a8e873f37aba8dcb; no voters:
I20260809 00:42:41.494524 7355 leader_election.cc:288] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:41.494576 7358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:41.494760 7358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 1 LEADER]: Becoming Leader. State: Replica: 1f8a3704ece84f08a8e873f37aba8dcb, State: Running, Role: LEADER
I20260809 00:42:41.494822 7355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:41.494948 7358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } }
I20260809 00:42:41.495239 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 7258
I20260809 00:42:41.495362 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:41.495347 7359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } }
I20260809 00:42:41.495460 7359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:41.495625 31139 external_mini_cluster.cc:1428] Setting key 2739e5c3717f35268c67ad6d91d98113
I20260809 00:42:41.495710 7361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f8a3704ece84f08a8e873f37aba8dcb. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } }
I20260809 00:42:41.495801 7361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:41.496237 7371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:41.496836 7371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:41.498503 7371 catalog_manager.cc:1347] Generated new cluster ID: 36fc3005ae8f4261a5790d5d99715fd0
I20260809 00:42:41.498559 7371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:42:41 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:41.508358 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:41.496303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.30.104.254:40589)
0809 00:42:41.496548 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:41.496558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:41.496596 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:42:41.496726 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:41.496733 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:41.496793 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:41.496958 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:41.497741 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.499029 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.500129 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.500747 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.500848 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:41.503788 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:41.503793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:41.503794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:41.503807 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:41.506351 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:41.506772 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:41.506775 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:41.506776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:41.506915 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:41.507125 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:41.507126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:41.507126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:41.507504 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:41.507667 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:41.507943 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 00:42:41.508107 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":85,"threads_started":1}
I20260809 00:42:41.517396 7371 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:41.518178 7371 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:41.522943 7371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: Generated new TSK 0
I20260809 00:42:41.523159 7371 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab.
Entry for principal oryx/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab.
I20260809 00:42:41.559610 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:40589
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab
--principal=oryx/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:41.651366 7398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:41.651677 7398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:41.651705 7398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:41.652944 7398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:41.652985 7398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:41.653014 7398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:41.654549 7398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab
--principal=oryx/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:41.654764 7398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:41.654963 7398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:41.660542 7404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:41.660534 7407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:41.660527 7409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:41.660567 7398 server_base.cc:1034] running on GCE node
I20260809 00:42:41.661047 7398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:41.661322 7398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:41.662506 7398 hybrid_clock.cc:648] HybridClock initialized: now 1786236161662492 us; error 32 us; skew 500 ppm
Aug 09 00:42:41 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:41.665465 7398 init.cc:377] Logged in from keytab as oryx/127.30.104.193@KRBTEST.COM (short username oryx)
I20260809 00:42:41.665961 7398 webserver.cc:460] Webserver started at http://127.30.104.193:39807/ using document root <none> and password file <none>
I20260809 00:42:41.666225 7398 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:41.666298 7398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:41.666453 7398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:41.667744 7398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "fb53d311bae2400096cd1463ff086f92"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "f719885e452c47f86e0d70c1ca4f1544"
server_key_iv: "237c07faa7174c01be00fc7188acb524"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.668140 7398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "fb53d311bae2400096cd1463ff086f92"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "f719885e452c47f86e0d70c1ca4f1544"
server_key_iv: "237c07faa7174c01be00fc7188acb524"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.669818 7398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:41.670724 7416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:41.671068 7398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:41.671196 7398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "fb53d311bae2400096cd1463ff086f92"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "f719885e452c47f86e0d70c1ca4f1544"
server_key_iv: "237c07faa7174c01be00fc7188acb524"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.671288 7398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:41.679006 7398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:41.681345 7398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:41.681548 7398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:41.681804 7398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:41.682153 7398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:41.682211 7398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:41.682255 7398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:41.682282 7398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:41.688691 7398 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:33821
I20260809 00:42:41.688732 7582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:33821 every 8 connection(s)
I20260809 00:42:41.689123 7398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:41.695947 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 7398
I20260809 00:42:41.696046 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
Aug 09 00:42:41 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:41.696269 31139 external_mini_cluster.cc:1428] Setting key dd33a2746f066dd244275aebe0653f6e
I20260809 00:42:41.699982 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:41.689888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:49511 (local address 127.30.104.254:40589)
0809 00:42:41.689977 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:41.689980 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:41.690164 (+ 184us) server_negotiation.cc:407] Connection header received
0809 00:42:41.690827 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:41.690829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:41.690845 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:41.690917 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:41.692097 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.692641 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.693733 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.693977 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.694053 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:41.696840 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:41.696842 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:41.696842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:41.696850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:41.698421 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:41.698890 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:41.698891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:41.698892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:41.698963 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:41.699244 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:41.699245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:41.699245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:41.699385 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:41.699501 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:41.699812 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 00:42:41.699895 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:42:41.700294 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:41.689847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:49511)
0809 00:42:41.690033 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:41.690072 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:41.690711 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:41.691008 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:41.691012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:41.691771 (+ 759us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:41.691984 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:41.691989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.692765 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.692766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:41.693616 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:41.693619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.694072 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.694073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:41.694176 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:41.694865 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:41.694874 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:41.696710 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:41.698534 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:41.698542 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:41.698545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:41.698793 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:41.699034 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:41.699035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:41.699035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:41.699167 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:41.699476 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:41.699484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:41.699648 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 00:42:41.699774 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 00:42:41.700000 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
I20260809 00:42:41.700886 7583 heartbeater.cc:347] Connected to a master server at 127.30.104.254:40589
I20260809 00:42:41.701010 7583 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:41.701215 7583 heartbeater.cc:511] Master 127.30.104.254:40589 requested a full tablet report, sending...
I20260809 00:42:41.701822 7305 ts_manager.cc:194] Registered new tserver with Master: fb53d311bae2400096cd1463ff086f92 (127.30.104.193:33821)
I20260809 00:42:41.702409 7305 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:49511
WARNING: no policy specified for oryx/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab.
Entry for principal oryx/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab.
I20260809 00:42:41.727916 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:40589
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab
--principal=oryx/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:41.816553 7589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:41.816838 7589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:41.816881 7589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:41.818177 7589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:41.818274 7589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:41.818324 7589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:41.820065 7589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab
--principal=oryx/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:41.820400 7589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:41.820681 7589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:41.826462 7596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:41.826622 7589 server_base.cc:1034] running on GCE node
W20260809 00:42:41.826471 7600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:41.826471 7598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:41.827033 7589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:41.827276 7589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:41.828435 7589 hybrid_clock.cc:648] HybridClock initialized: now 1786236161828415 us; error 44 us; skew 500 ppm
Aug 09 00:42:41 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:41.831319 7589 init.cc:377] Logged in from keytab as oryx/127.30.104.194@KRBTEST.COM (short username oryx)
I20260809 00:42:41.831866 7589 webserver.cc:460] Webserver started at http://127.30.104.194:42505/ using document root <none> and password file <none>
I20260809 00:42:41.832149 7589 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:41.832198 7589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:41.832345 7589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:41.833475 7589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "b6da8911a57e4f19958a580c25f21555"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "07eaad8685325a813e312118f1f9af32"
server_key_iv: "fb14dbaa403c6e204b47430cae1ea561"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.833858 7589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "b6da8911a57e4f19958a580c25f21555"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "07eaad8685325a813e312118f1f9af32"
server_key_iv: "fb14dbaa403c6e204b47430cae1ea561"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.835289 7589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:41.836175 7608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:41.836397 7589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:41.836513 7589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "b6da8911a57e4f19958a580c25f21555"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "07eaad8685325a813e312118f1f9af32"
server_key_iv: "fb14dbaa403c6e204b47430cae1ea561"
server_key_version: "encryptionkey@0"
I20260809 00:42:41.836611 7589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:41.854774 7589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:41.857380 7589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:41.857615 7589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:41.857908 7589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:41.858270 7589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:41.858330 7589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:41.858376 7589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:41.858413 7589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:41.865168 7589 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:44235
I20260809 00:42:41.865224 7771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:44235 every 8 connection(s)
I20260809 00:42:41.865610 7589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:42:41 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:41.874967 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 7589
I20260809 00:42:41.875121 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:41.875339 31139 external_mini_cluster.cc:1428] Setting key 2dc087acaf1870ab141b0b32dbd38518
I20260809 00:42:41.876788 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:41.866503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:55733 (local address 127.30.104.254:40589)
0809 00:42:41.866592 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:41.866594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:41.866801 (+ 207us) server_negotiation.cc:407] Connection header received
0809 00:42:41.867510 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:41.867512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:41.867526 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:41.867598 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:41.868739 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.869267 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.870328 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.870557 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.870619 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:41.873158 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:41.873160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:41.873160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:41.873168 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:41.875052 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:41.875609 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:41.875611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:41.875611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:41.875677 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:41.875994 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:41.875995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:41.875995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:41.876157 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:41.876239 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:41.876532 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 00:42:41.876638 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 00:42:41.876801 7774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:41.866469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:55733)
0809 00:42:41.866666 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:41.866703 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:41.867300 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:41.867692 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:41.867695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:41.868360 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:41.868571 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:41.868576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.869368 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.869369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:41.870238 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:41.870241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:41.870638 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:41.870639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:41.870750 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:41.871271 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:41.871291 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:41.873029 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:41.875174 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:41.875186 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:41.875189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:41.875515 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:41.875777 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:41.875778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:41.875779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:41.875914 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:41.876237 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:41.876243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:41.876379 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 00:42:41.876477 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 00:42:41.876620 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
I20260809 00:42:41.877339 7772 heartbeater.cc:347] Connected to a master server at 127.30.104.254:40589
I20260809 00:42:41.877476 7772 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:41.877741 7772 heartbeater.cc:511] Master 127.30.104.254:40589 requested a full tablet report, sending...
I20260809 00:42:41.878165 7305 ts_manager.cc:194] Registered new tserver with Master: b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235)
I20260809 00:42:41.878697 7305 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:55733
WARNING: no policy specified for oryx/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab.
Entry for principal oryx/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab.
I20260809 00:42:41.906838 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:40589
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab
--principal=oryx/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:41.997957 7779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:41.998200 7779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:41.998229 7779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:41.999665 7779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:41.999722 7779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:41.999754 7779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:42.001439 7779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab
--principal=oryx/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:42.001744 7779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:42.002007 7779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:42.007737 7784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:42.007840 7779 server_base.cc:1034] running on GCE node
W20260809 00:42:42.007869 7790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.007869 7785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:42.008262 7779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:42.008566 7779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:42.009752 7779 hybrid_clock.cc:648] HybridClock initialized: now 1786236162009720 us; error 49 us; skew 500 ppm
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:42.012907 7779 init.cc:377] Logged in from keytab as oryx/127.30.104.195@KRBTEST.COM (short username oryx)
I20260809 00:42:42.013455 7779 webserver.cc:460] Webserver started at http://127.30.104.195:34303/ using document root <none> and password file <none>
I20260809 00:42:42.013762 7779 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:42.013815 7779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:42.013916 7779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:42.015043 7779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "24133fd916f249f291f167baf4e2974a"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c91502858866365013a3614c9fb678a1"
server_key_iv: "6524d6de3d6d2c86c5d29b99dbc20f0d"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.015374 7779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "24133fd916f249f291f167baf4e2974a"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c91502858866365013a3614c9fb678a1"
server_key_iv: "6524d6de3d6d2c86c5d29b99dbc20f0d"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.017010 7779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:42.017992 7803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.018266 7779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:42.018410 7779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "24133fd916f249f291f167baf4e2974a"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c91502858866365013a3614c9fb678a1"
server_key_iv: "6524d6de3d6d2c86c5d29b99dbc20f0d"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.018509 7779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:42.088533 7779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:42.091061 7779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:42.091274 7779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:42.091651 7779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:42.092044 7779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:42.092109 7779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.092160 7779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:42.092190 7779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.099223 7779 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:33781
I20260809 00:42:42.099269 7954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:33781 every 8 connection(s)
I20260809 00:42:42.099716 7779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:42.105356 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 7779
I20260809 00:42:42.105455 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:42.105629 31139 external_mini_cluster.cc:1428] Setting key e33f28afa24c1c7a39894b66b59c528b
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:42.110638 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.100589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:42407 (local address 127.30.104.254:40589)
0809 00:42:42.100692 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.100695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.100880 (+ 185us) server_negotiation.cc:407] Connection header received
0809 00:42:42.101401 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.101402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.101415 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.101483 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.102509 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.103107 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.104245 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.104472 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.104541 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.107372 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.107373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.107374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.107381 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.108960 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.109384 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.109385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.109386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.109453 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.109822 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.109823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.109824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.110028 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.110111 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.110475 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.110563 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:42:42.110802 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.100543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:42407)
0809 00:42:42.100742 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.100782 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.101280 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.101569 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.101572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.102190 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.102400 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.102405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.103228 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.103228 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.104136 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.104138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.104568 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.104569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.104683 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.105289 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.105298 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.107276 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.109051 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.109058 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.109061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.109310 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.109540 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.109541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.109541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.109657 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.110150 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.110157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.110301 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.110411 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.110556 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":128,"threads_started":1}
I20260809 00:42:42.111545 7955 heartbeater.cc:347] Connected to a master server at 127.30.104.254:40589
I20260809 00:42:42.111656 7955 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:42.112030 7955 heartbeater.cc:511] Master 127.30.104.254:40589 requested a full tablet report, sending...
I20260809 00:42:42.112622 7305 ts_manager.cc:194] Registered new tserver with Master: 24133fd916f249f291f167baf4e2974a (127.30.104.195:33781)
I20260809 00:42:42.113116 7305 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:42407
I20260809 00:42:42.118682 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.104.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.253.keytab.
Entry for principal oryx/127.30.104.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.253.keytab.
I20260809 00:42:42.151153 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.253:37041
--webserver_interface=127.30.104.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.253.keytab
--principal=oryx/127.30.104.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:40589,127.30.104.253:37041
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:42.239346 7962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:42.239651 7962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:42.239677 7962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:42.240955 7962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:42.241007 7962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:42.241022 7962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:42.241035 7962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:42.241048 7962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:42.242575 7962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:40589,127.30.104.253:37041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.253.keytab
--principal=oryx/127.30.104.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.253:37041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.104.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
W20260809 00:42:42.242692 7962 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.104.254:40589,127.30.104.253:37041'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 00:42:42.242818 7962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:42.243010 7962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:42.248436 7969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.248452 7972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.248431 7970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:42.248767 7962 server_base.cc:1034] running on GCE node
I20260809 00:42:42.249058 7962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:42.249398 7962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:42.250552 7962 hybrid_clock.cc:648] HybridClock initialized: now 1786236162250528 us; error 39 us; skew 500 ppm
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:42.253795 7962 init.cc:377] Logged in from keytab as oryx/127.30.104.253@KRBTEST.COM (short username oryx)
I20260809 00:42:42.254374 7962 webserver.cc:460] Webserver started at http://127.30.104.253:34147/ using document root <none> and password file <none>
I20260809 00:42:42.254650 7962 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:42.254729 7962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:42.254894 7962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:42.256098 7962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data/instance:
uuid: "82e929c186674536a7841a1dc3b681b5"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "1918a9caba00032a0a4dec66d9a97704"
server_key_iv: "6821d5b5964891d272b39d1523c4e2f9"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.256488 7962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal/instance:
uuid: "82e929c186674536a7841a1dc3b681b5"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "1918a9caba00032a0a4dec66d9a97704"
server_key_iv: "6821d5b5964891d272b39d1523c4e2f9"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.257944 7962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:42.258798 7981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.258988 7962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:42.259112 7962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
uuid: "82e929c186674536a7841a1dc3b681b5"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "1918a9caba00032a0a4dec66d9a97704"
server_key_iv: "6821d5b5964891d272b39d1523c4e2f9"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.259193 7962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:42.277400 7962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:42.280031 7962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:42.280222 7962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:42.284706 7962 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.253:37041
I20260809 00:42:42.284745 8056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.253:37041 every 8 connection(s)
I20260809 00:42:42.285133 7962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
I20260809 00:42:42.286239 8057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:42.288158 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 7962
I20260809 00:42:42.287978 8057 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.253@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.104.253@KRBTEST.COM
I20260809 00:42:42.300005 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.289147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42438 (local address 127.30.104.254:40589)
0809 00:42:42.289210 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.289213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.289398 (+ 185us) server_negotiation.cc:407] Connection header received
0809 00:42:42.290126 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.290127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.290142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.290225 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.290813 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.291319 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.292330 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.292536 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.292629 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.296049 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.296051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.296051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.296061 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.297982 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.298655 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.298656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.298657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.298733 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.299024 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.299026 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.299026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.299256 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.299344 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.299801 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.299921 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:42:42.300179 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.288805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42438)
0809 00:42:42.289256 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.289307 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.290011 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.290251 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.290254 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.290542 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.290708 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.290713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.291492 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.291494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.292232 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.292234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.292642 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.292643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.292776 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.293411 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.293422 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.295859 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.298083 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.298090 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.298093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.298508 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.298822 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.298823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.298823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.298942 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.299355 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.299361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.299598 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.299706 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.299905 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":114,"threads_started":1}
I20260809 00:42:42.301019 8057 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:42:42.301707 8063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.288980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47134 (local address 127.30.104.253:37041)
0809 00:42:42.289256 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.289266 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.289515 (+ 249us) server_negotiation.cc:407] Connection header received
0809 00:42:42.290025 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.290037 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.290062 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.290174 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.290862 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.291841 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.292669 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.293059 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.293138 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.297421 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.297433 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.297433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.297443 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.299743 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.300245 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.300248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.300249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.300320 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.300784 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.300786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.300786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.300986 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.301179 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.301460 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.301554 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":146,"threads_started":1}
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.253@KRBTEST.COM for oryx/127.30.104.253@KRBTEST.COM
I20260809 00:42:42.309103 8062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.301328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.0.0.1:47138)
0809 00:42:42.301423 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.301681 (+ 258us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.301734 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.301982 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.301983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.302124 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.302171 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.302173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.302911 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.302912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.303416 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.303417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.303886 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.303887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.303963 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.304522 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.304527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.305888 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.308060 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.308061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.308061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.308235 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.308418 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.308419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.308419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.308487 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.308784 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.308785 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.308832 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.308901 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.308998 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 00:42:42.309183 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.301462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.30.104.253:37041)
0809 00:42:42.301707 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.301708 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.301718 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:42.301894 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.301895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.301907 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.301969 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.302257 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.302817 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.303570 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.303799 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.303887 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.305998 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.305999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.305999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.306006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.307976 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.308303 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.308304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.308304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.308358 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.308542 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.308543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.308543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.308657 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.308711 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.308929 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.309052 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":223,"thread_start_us":185,"threads_started":1}
I20260809 00:42:42.310772 8057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Bootstrap starting.
I20260809 00:42:42.311409 8057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:42.311743 8057 log.cc:824] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:42.312542 8057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: No bootstrap required, opened a new log
I20260809 00:42:42.313930 8057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } }
I20260809 00:42:42.314062 8057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:42.314085 8057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82e929c186674536a7841a1dc3b681b5, State: Initialized, Role: FOLLOWER
I20260809 00:42:42.314220 8057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } }
I20260809 00:42:42.314615 8057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:42.314584 8066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } } }
I20260809 00:42:42.314803 8066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:42:42.316488 8080 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:42:42.316552 8080 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:42:42.318409 7962 master_runner.cc:428] Detected that this master 82e929c186674536a7841a1dc3b681b5 is joining an existing cluster
I20260809 00:42:42.318568 7962 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.104.253:37041
I20260809 00:42:42.326848 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.319881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.30.104.254:40589)
0809 00:42:42.319932 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.319935 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.320096 (+ 161us) server_negotiation.cc:407] Connection header received
0809 00:42:42.320169 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.320170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.320182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.320236 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.320797 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.321599 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.322752 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.323040 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.323110 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.324107 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.324109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.324109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.324116 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.325551 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.325946 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.325948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.325948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.326012 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.326226 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.326227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.326227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.326377 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.326466 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.326644 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.326736 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:42:42.326905 8089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.319779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42440)
0809 00:42:42.320040 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.320049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.320101 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.320297 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.320298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.320567 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.320685 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.320687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.321733 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.321734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.322655 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.322658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.323107 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.323108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.323174 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.323633 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.323638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.324026 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.325641 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.325642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.325643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.325870 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.326088 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.326089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.326090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.326152 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.326450 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.326451 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.326514 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.326600 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.326758 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":31,"thread_start_us":107,"threads_started":1}
I20260809 00:42:42.327648 8087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.319780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.0.0.1:47144)
0809 00:42:42.320159 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.320166 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.320224 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.320363 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.320363 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.320574 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.320671 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.320674 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.321477 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.321478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.322056 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.322058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.322540 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.322541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.322620 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.323288 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.323293 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.323869 (+ 576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.326532 (+ 2663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.326533 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.326533 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.326787 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.326982 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.326982 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.326983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.327044 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.327282 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.327283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.327317 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.327388 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.327529 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":36,"thread_start_us":95,"threads_started":1}
I20260809 00:42:42.327667 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.319974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47144 (local address 127.30.104.253:37041)
0809 00:42:42.320033 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.320035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.320221 (+ 186us) server_negotiation.cc:407] Connection header received
0809 00:42:42.320279 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.320280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.320291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.320358 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.320761 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.321375 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.322143 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.322447 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.322507 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.323967 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.323968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.323968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.323977 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.326448 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.326864 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.326865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.326865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.326914 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.327106 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.327107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.327107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.327212 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.327248 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.327424 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.327584 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":40}
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.254@KRBTEST.COM for oryx/127.30.104.253@KRBTEST.COM
I20260809 00:42:42.338183 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.329775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47146 (local address 127.30.104.253:37041)
0809 00:42:42.329818 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.329820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.330032 (+ 212us) server_negotiation.cc:407] Connection header received
0809 00:42:42.330090 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.330091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.330101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.330158 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.330685 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.331192 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.332380 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.332700 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.332771 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.335333 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.335334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.335335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.335342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.336933 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.337406 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.337407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.337407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.337464 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.337676 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.337676 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.337677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.337785 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.337837 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.338052 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.338116 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:42:42.338294 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.329689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.0.0.1:47146)
0809 00:42:42.329877 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.329930 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.330026 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.330166 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.330169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.330528 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.330594 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.330596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.331278 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.331280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.332286 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.332290 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.332794 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.332795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.332905 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.333433 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.333439 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.335234 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.337021 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.337030 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.337033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.337321 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.337537 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.337538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.337539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.337611 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.337851 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.337861 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.337925 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.338033 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.338169 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260809 00:42:42.339010 7305 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.104.253:37041: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "82e929c186674536a7841a1dc3b681b5"
member_type: NON_VOTER
last_known_addr {
host: "127.30.104.253"
port: 37041
}
attrs {
promote: true
}
}
dest_uuid: "1f8a3704ece84f08a8e873f37aba8dcb"
cas_config_opid_index: -1
I20260809 00:42:42.339393 7305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } }
I20260809 00:42:42.340020 8095 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 82e929c186674536a7841a1dc3b681b5 (127.30.104.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } }
I20260809 00:42:42.340416 8020 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:42.341238 8095 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.104.253:37041
I20260809 00:42:42.341434 8020 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Refusing update from remote peer 1f8a3704ece84f08a8e873f37aba8dcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 00:42:42.341539 7962 master.cc:537] Master@127.30.104.253:37041 shutting down...
I20260809 00:42:42.341991 8095 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:42:42.342760 7284 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260809 00:42:42.342934 8093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } }
I20260809 00:42:42.343122 8093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
W20260809 00:42:42.343582 7284 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb -> Peer 82e929c186674536a7841a1dc3b681b5 (127.30.104.253:37041): Couldn't send request to peer 82e929c186674536a7841a1dc3b681b5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 00:42:42.344535 7962 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:42.344648 7962 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:42.344704 7962 tablet_replica.cc:330] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: stopping tablet replica
I20260809 00:42:42.357771 7962 master.cc:552] Master@127.30.104.253:37041 shutdown complete.
I20260809 00:42:42.357842 7962 master_runner.cc:305] Clearing existing system tablet
I20260809 00:42:42.357995 7962 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:42.358048 7962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:42.358856 7962 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:42.359565 8099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.359754 7962 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:42.359824 7962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:42.359853 7962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
uuid: "82e929c186674536a7841a1dc3b681b5"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "1918a9caba00032a0a4dec66d9a97704"
server_key_iv: "6821d5b5964891d272b39d1523c4e2f9"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.360695 7962 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:42.362560 7962 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 00:42:42.362636 7962 log.cc:1196] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 00:42:42.362843 7962 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Deleting consensus metadata
I20260809 00:42:42.362982 7962 master_runner.cc:315] Copying system tablet from 127.30.104.254:40589
I20260809 00:42:42.363662 7962 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: tablet copy: Beginning tablet copy session from remote peer at address 127.30.104.254:40589
I20260809 00:42:42.370666 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.364085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.30.104.254:40589)
0809 00:42:42.364169 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.364172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.364268 (+ 96us) server_negotiation.cc:407] Connection header received
0809 00:42:42.364351 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.364352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.364362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.364417 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.364790 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.365376 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.366448 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.366639 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.366711 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.367934 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.367935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.367936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.367943 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.369539 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.369947 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.369948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.369948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.370009 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.370214 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.370215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.370215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.370323 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.370375 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.370534 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.370597 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:42:42.370745 8105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.364044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42444)
0809 00:42:42.364196 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.364208 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.364285 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.364422 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.364423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.364637 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.364710 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.364713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.365469 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.365470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.366346 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.366348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.366723 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.366724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.366794 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.367281 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.367287 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.367764 (+ 477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.369631 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.369632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.369633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.369867 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.370080 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.370081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.370081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.370150 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.370387 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.370388 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.370440 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.370518 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.370629 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":105,"threads_started":1}
I20260809 00:42:42.371325 7330 tablet_copy_service.cc:140] P 1f8a3704ece84f08a8e873f37aba8dcb: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 82e929c186674536a7841a1dc3b681b5 ({username='oryx', principal='oryx/127.30.104.253@KRBTEST.COM'} at 127.0.0.1:42444)
I20260809 00:42:42.371435 7330 tablet_copy_service.cc:161] P 1f8a3704ece84f08a8e873f37aba8dcb: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 82e929c186674536a7841a1dc3b681b5 at {username='oryx', principal='oryx/127.30.104.253@KRBTEST.COM'} at 127.0.0.1:42444: session id = 82e929c186674536a7841a1dc3b681b5-00000000000000000000000000000000
I20260809 00:42:42.372239 7330 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:42:42.372931 7962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 00:42:42.374315 7962 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: tablet copy: Starting download of 0 data blocks...
I20260809 00:42:42.374486 7962 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: tablet copy: Starting download of 1 WAL segments...
I20260809 00:42:42.375461 7962 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:42:42.376502 7330 tablet_copy_service.cc:342] P 1f8a3704ece84f08a8e873f37aba8dcb: Request end of tablet copy session 82e929c186674536a7841a1dc3b681b5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.104.253@KRBTEST.COM'} at 127.0.0.1:42444
I20260809 00:42:42.376600 7330 tablet_copy_service.cc:433] P 1f8a3704ece84f08a8e873f37aba8dcb: ending tablet copy session 82e929c186674536a7841a1dc3b681b5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 82e929c186674536a7841a1dc3b681b5
I20260809 00:42:42.379406 7962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:42.380713 8106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.380784 8109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.380892 8107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:42.381032 7962 server_base.cc:1034] running on GCE node
I20260809 00:42:42.381136 7962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:42.381306 7962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:42.382476 7962 hybrid_clock.cc:648] HybridClock initialized: now 1786236162382464 us; error 30 us; skew 500 ppm
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:42.383972 7962 init.cc:377] Logged in from keytab as oryx/127.30.104.253@KRBTEST.COM (short username oryx)
I20260809 00:42:42.384284 7962 webserver.cc:460] Webserver started at http://127.30.104.253:36341/ using document root <none> and password file <none>
I20260809 00:42:42.384475 7962 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:42.384508 7962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:42.385156 7962 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:42.385710 8122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.385833 7962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:42.385882 7962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
uuid: "82e929c186674536a7841a1dc3b681b5"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "1918a9caba00032a0a4dec66d9a97704"
server_key_iv: "6821d5b5964891d272b39d1523c4e2f9"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.385946 7962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:42.402156 7962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:42.404264 7962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:42.407835 7962 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.253:37041
I20260809 00:42:42.408072 8196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.253:37041 every 8 connection(s)
I20260809 00:42:42.408280 7962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
I20260809 00:42:42.409054 8198 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:42:42.409507 8198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Bootstrap starting.
I20260809 00:42:42.412093 8198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:42:42.412352 8198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Bootstrap complete.
I20260809 00:42:42.412575 8198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } }
I20260809 00:42:42.412671 8198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 82e929c186674536a7841a1dc3b681b5, State: Initialized, Role: LEARNER
I20260809 00:42:42.412725 8198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } }
I20260809 00:42:42.413007 8198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:42.412968 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } }
I20260809 00:42:42.413058 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: This master's current role is: LEARNER
I20260809 00:42:42.414675 8216 catalog_manager.cc:1259] Loaded cluster ID: 36fc3005ae8f4261a5790d5d99715fd0
I20260809 00:42:42.414757 8216 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: loading cluster ID for follower catalog manager: success
I20260809 00:42:42.415881 8216 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: acquiring CA information for follower catalog manager: success
I20260809 00:42:42.416193 8216 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:42:42.461719 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.454042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47184 (local address 127.30.104.253:37041)
0809 00:42:42.454217 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.454219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.454228 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:42.454261 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.454261 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.454270 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.454352 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.454971 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.455869 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.456749 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.456997 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.457096 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.458632 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.458634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.458634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.458643 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.460299 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.460754 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.460756 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.460756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.460830 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.461065 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.461066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.461066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.461192 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.461301 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.461467 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.461565 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for oryx/127.30.104.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.252.keytab.
Entry for principal oryx/127.30.104.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.252.keytab.
I20260809 00:42:42.490797 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.252:45939
--webserver_interface=127.30.104.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.252.keytab
--principal=oryx/127.30.104.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:40589,127.30.104.253:37041,127.30.104.252:45939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:42.579514 8221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:42.579768 8221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:42.579807 8221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:42.581053 8221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:42.581118 8221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:42.581144 8221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:42.581172 8221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:42.581202 8221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:42.582824 8221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:40589,127.30.104.253:37041,127.30.104.252:45939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.252.keytab
--principal=oryx/127.30.104.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.252:45939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.104.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:42.583129 8221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:42.583385 8221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:42.589121 8236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.589160 8238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.589143 8235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:42.589207 8221 server_base.cc:1034] running on GCE node
I20260809 00:42:42.589641 8221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:42.589991 8221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:42.591161 8221 hybrid_clock.cc:648] HybridClock initialized: now 1786236162591122 us; error 59 us; skew 500 ppm
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:42.594112 8221 init.cc:377] Logged in from keytab as oryx/127.30.104.252@KRBTEST.COM (short username oryx)
I20260809 00:42:42.594610 8221 webserver.cc:460] Webserver started at http://127.30.104.252:45611/ using document root <none> and password file <none>
I20260809 00:42:42.594944 8221 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:42.595042 8221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:42.595222 8221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:42.596578 8221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data/instance:
uuid: "406dea23b8464ffd88e6881c301c3094"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c0c0ca6e9085177fc26eb1d0a2727ec3"
server_key_iv: "587a649485d480dc54bae3912d50a3c2"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.597008 8221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal/instance:
uuid: "406dea23b8464ffd88e6881c301c3094"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c0c0ca6e9085177fc26eb1d0a2727ec3"
server_key_iv: "587a649485d480dc54bae3912d50a3c2"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.598448 8221 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:42.599288 8246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.599532 8221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:42.599640 8221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
uuid: "406dea23b8464ffd88e6881c301c3094"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c0c0ca6e9085177fc26eb1d0a2727ec3"
server_key_iv: "587a649485d480dc54bae3912d50a3c2"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.599730 8221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:42.616771 8221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:42.619247 8221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:42.619469 8221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:42.624001 8221 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.252:45939
I20260809 00:42:42.624044 8318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.252:45939 every 8 connection(s)
I20260809 00:42:42.624451 8221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
I20260809 00:42:42.625598 8320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:42.627259 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 8221
I20260809 00:42:42.627305 8320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.252@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.104.252@KRBTEST.COM
I20260809 00:42:42.638979 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.628301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.30.104.254:40589)
0809 00:42:42.628343 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.628345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.628539 (+ 194us) server_negotiation.cc:407] Connection header received
0809 00:42:42.629061 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.629063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.629077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.629147 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.629979 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.630762 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.632078 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.632394 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.632469 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.635257 (+ 2788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.635259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.635260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.635267 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.637178 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.637753 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.637755 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.637755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.637824 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.638131 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.638132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.638132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.638303 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.638372 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.638775 (+ 403us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.638866 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:42.639498 8323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.628415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56698 (local address 127.30.104.252:45939)
0809 00:42:42.628594 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.628599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.628637 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:42:42.628977 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.628988 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.629025 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.629141 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.629975 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.631094 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.632130 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.632620 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.632679 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.635846 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.635854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.635855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.635862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.637746 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.638215 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.638219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.638219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.638276 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.638578 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.638579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.638579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.638751 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.638915 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.639087 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.639216 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260809 00:42:42.639515 8322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.628195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42448)
0809 00:42:42.628384 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.628430 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.628961 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.629205 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.629208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.629627 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.629871 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.629877 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.630873 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.630873 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.631982 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.631985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.632509 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.632511 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.632615 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.633232 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.633243 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.635132 (+ 1889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.637295 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.637302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.637305 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.637649 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.637895 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.637895 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.637896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.638046 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.638416 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.638424 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.638627 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.638744 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.638939 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260809 00:42:42.640038 8320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.252@KRBTEST.COM for oryx/127.30.104.253@KRBTEST.COM
I20260809 00:42:42.646497 8322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.640320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.0.0.1:47194)
0809 00:42:42.640371 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.640382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.640419 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.640588 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.640589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.640704 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.640738 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.640739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.641514 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.641514 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.642017 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.642019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.642407 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.642407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.642451 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.642910 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.642922 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.644157 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.645660 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.645661 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.645662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.645828 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.646017 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.646018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.646018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.646070 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.646255 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.646256 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.646293 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.646343 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.646419 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:42:42.646557 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.640410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47194 (local address 127.30.104.253:37041)
0809 00:42:42.640460 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.640462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.640470 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:42.640501 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.640502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.640513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.640582 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.640837 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.641423 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.642127 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.642331 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.642402 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.644241 (+ 1839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.644243 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.644243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.644251 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.645580 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.645903 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.645904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.645904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.645956 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.646109 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.646109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.646110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.646207 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.646252 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.646360 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.646468 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:42:42.647007 8320 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.252@KRBTEST.COM for oryx/127.30.104.252@KRBTEST.COM
I20260809 00:42:42.653535 8322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.647291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.0.0.1:56712)
0809 00:42:42.647345 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.647358 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.647406 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.647593 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.647593 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.647708 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.647748 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.647750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.648443 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.648444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.648922 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.648923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.649300 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.649301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.649338 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.649794 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.649799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.651023 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.652582 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.652583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.652583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.652748 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.652981 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.652982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.652982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.653042 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.653273 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.653273 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.653309 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.653359 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.653441 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:42:42.653584 8323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.647378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.30.104.252:45939)
0809 00:42:42.647429 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.647465 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.647473 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:42.647524 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.647525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.647536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.647590 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.647839 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.648324 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.649002 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.649229 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.649293 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.651104 (+ 1811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.651106 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.651106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.651113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.652506 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.652834 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.652835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.652835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.652895 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.653103 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.653104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.653104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.653215 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.653269 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.653400 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.653490 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":34}
I20260809 00:42:42.655112 8320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Bootstrap starting.
I20260809 00:42:42.656060 8320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:42.656366 8320 log.cc:824] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:42.657269 8320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: No bootstrap required, opened a new log
I20260809 00:42:42.658802 8320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } }
I20260809 00:42:42.658936 8320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:42.658957 8320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 406dea23b8464ffd88e6881c301c3094, State: Initialized, Role: FOLLOWER
I20260809 00:42:42.659106 8320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } }
I20260809 00:42:42.659528 8320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:42.659535 8325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } } }
I20260809 00:42:42.659696 8325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:42:42.661466 8341 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:42:42.661537 8341 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:42:42.663887 8221 master_runner.cc:428] Detected that this master 406dea23b8464ffd88e6881c301c3094 is joining an existing cluster
I20260809 00:42:42.663973 8221 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.104.252:45939
I20260809 00:42:42.673230 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.665043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42464)
0809 00:42:42.665246 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.665258 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.665342 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.665523 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.665524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.665837 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.665934 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.665937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.666944 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.666945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.667908 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.667911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.668488 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.668489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.668553 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.669066 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.669071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.669592 (+ 521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.671909 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.671910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.671911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.672152 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.672400 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.672400 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.672401 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.672474 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.672854 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.672855 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.672904 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.672994 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.673111 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":77,"threads_started":1}
I20260809 00:42:42.673264 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.665149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.30.104.254:40589)
0809 00:42:42.665255 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.665258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.665315 (+ 57us) server_negotiation.cc:407] Connection header received
0809 00:42:42.665403 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.665404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.665415 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.665469 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.666030 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.666842 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.668032 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.668346 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.668391 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.669702 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.669703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.669704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.669713 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.671811 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.672257 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.672259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.672259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.672326 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.672567 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.672569 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.672569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.672760 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.672852 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.673052 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.673154 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 00:42:42.674536 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.665205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.30.104.253:37041)
0809 00:42:42.665297 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.665299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.665531 (+ 232us) server_negotiation.cc:407] Connection header received
0809 00:42:42.665638 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.665639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.665650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.665679 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.667348 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.668237 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.669513 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.669811 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.669893 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.671063 (+ 1170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.671066 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.671066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.671077 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.672989 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.673527 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.673529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.673529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.673605 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.673904 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.673905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.673906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.674066 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.674141 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.674347 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.674438 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 00:42:42.674885 8352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.665081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.0.0.1:47202)
0809 00:42:42.665473 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.665483 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.665565 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.665750 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.665751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.667090 (+ 1339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.667159 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.667161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.668334 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.668335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.669395 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.669398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.669904 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.669906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.669994 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.670495 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.670500 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.670936 (+ 436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.673107 (+ 2171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.673108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.673108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.673394 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.673731 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.673731 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.673732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.673817 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.674172 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.674173 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.674233 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.674633 (+ 400us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.674772 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":91,"threads_started":1}
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236161, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.254@KRBTEST.COM for oryx/127.30.104.252@KRBTEST.COM
I20260809 00:42:42.691200 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 7398
I20260809 00:42:42.691550 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.676072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.0.0.1:56734)
0809 00:42:42.676133 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.676145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.676188 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.676498 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.676499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.676685 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.676731 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.676733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.677511 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.677512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.678105 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.678107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.678560 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.678562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.678644 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.679132 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.679136 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.680726 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.690261 (+ 9535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.690262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.690263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.690509 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.690756 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.690756 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.690757 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.690821 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.691178 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.691179 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.691233 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.691281 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.691410 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:42:42.691901 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.665353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.0.0.1:56724)
0809 00:42:42.665479 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.665489 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.665569 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.665721 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.665721 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.666046 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.666107 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.666109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.666949 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.666950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.668136 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.668139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.668703 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.668704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.668767 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.688094 (+ 19327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.688102 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.688693 (+ 591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.690735 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.690735 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.690736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.690939 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.691150 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.691150 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.691150 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.691205 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.691506 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.691506 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.691544 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.691601 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.691722 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260809 00:42:42.692056 8353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.676150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.30.104.252:45939)
0809 00:42:42.676354 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.676356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.676362 (+ 6us) server_negotiation.cc:407] Connection header received
0809 00:42:42.676400 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.676401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.676410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.676492 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.676818 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.677402 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.678204 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.678449 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.678557 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.680863 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.680865 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.680865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.680873 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.690133 (+ 9260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.690609 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.690610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.690610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.690678 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.690889 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.690890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.690890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.691090 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.691735 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.691789 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.691876 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":123,"threads_started":1}
I20260809 00:42:42.692299 8323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.665276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56724 (local address 127.30.104.252:45939)
0809 00:42:42.665315 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.665318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.665538 (+ 220us) server_negotiation.cc:407] Connection header received
0809 00:42:42.665632 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.665633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.665640 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.665667 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.666193 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.666866 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.668233 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.668558 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.668603 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.688822 (+ 20219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.688823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.688823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.688831 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.690639 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.691027 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.691028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.691029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.691086 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.691264 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.691265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.691265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.691397 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.691484 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.692072 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.692165 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:42.709558 7305 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.104.252:45939: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "406dea23b8464ffd88e6881c301c3094"
member_type: NON_VOTER
last_known_addr {
host: "127.30.104.252"
port: 45939
}
attrs {
promote: true
}
}
dest_uuid: "1f8a3704ece84f08a8e873f37aba8dcb"
cas_config_opid_index: 5
I20260809 00:42:42.709914 7305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } }
I20260809 00:42:42.710487 8095 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 406dea23b8464ffd88e6881c301c3094 (127.30.104.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.711140 8281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:42.711647 8095 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.104.252:45939
I20260809 00:42:42.711731 8095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.711792 8095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:42.712127 8221 master.cc:537] Master@127.30.104.252:45939 shutting down...
I20260809 00:42:42.712237 8281 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Refusing update from remote peer 1f8a3704ece84f08a8e873f37aba8dcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 00:42:42.712558 8093 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 00:42:42.713512 7283 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb -> Peer 406dea23b8464ffd88e6881c301c3094 (127.30.104.252:45939): Couldn't send request to peer 406dea23b8464ffd88e6881c301c3094. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 00:42:42.715582 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:33821
--local_ip_for_outbound_sockets=127.30.104.193
--tserver_master_addrs=127.30.104.254:40589
--webserver_port=39807
--webserver_interface=127.30.104.193
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab
--principal=oryx/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041,127.30.104.252:45939 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:42:42.716547 8221 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:42.716656 8221 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:42.716702 8221 tablet_replica.cc:330] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: stopping tablet replica
I20260809 00:42:42.718672 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.710630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.0.0.1:47208)
0809 00:42:42.713979 (+ 3349us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.713992 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.714048 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.715025 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.715026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.715028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:42.715081 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.715083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.716105 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.716106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.717358 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.717362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.718328 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.718329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.718405 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.718439 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.718495 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.718549 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3311}
I20260809 00:42:42.719014 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.710748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47208 (local address 127.30.104.253:37041)
0809 00:42:42.710797 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.710800 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.714035 (+ 3235us) server_negotiation.cc:407] Connection header received
0809 00:42:42.714791 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.714793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.714820 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.714884 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:42.715184 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.716001 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.717468 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.718123 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.718194 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.718287 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.718853 (+ 566us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.718908 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:42:42.719260 8158 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 LEARNER]: Refusing update from remote peer 1f8a3704ece84f08a8e873f37aba8dcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 00:42:42.719529 8354 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 00:42:42.720211 8158 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 406dea23b8464ffd88e6881c301c3094 (127.30.104.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.721103 8221 master.cc:552] Master@127.30.104.252:45939 shutdown complete.
I20260809 00:42:42.721143 8221 master_runner.cc:305] Clearing existing system tablet
I20260809 00:42:42.721280 8221 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:42.721328 8221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:42.721489 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f8a3704ece84f08a8e873f37aba8dcb. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.721580 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: This master's current role is: LEARNER
I20260809 00:42:42.721621 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: NON_VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: true } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.721660 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: This master's current role is: LEARNER
I20260809 00:42:42.722338 8093 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: attempting to promote NON_VOTER 82e929c186674536a7841a1dc3b681b5 to VOTER
I20260809 00:42:42.722354 8221 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.722606 8093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } }
I20260809 00:42:42.723006 8362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.723160 8221 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:42.723214 8221 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:42.723239 8221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
uuid: "406dea23b8464ffd88e6881c301c3094"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c0c0ca6e9085177fc26eb1d0a2727ec3"
server_key_iv: "587a649485d480dc54bae3912d50a3c2"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.723363 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.723229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.0.0.1:56742)
0809 00:42:42.723276 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.723325 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:45939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 00:42:42.723629 8158 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 LEARNER]: Refusing update from remote peer 1f8a3704ece84f08a8e873f37aba8dcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 00:42:42.723896 8095 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 00:42:42.724201 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.724107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.0.0.1:56752)
0809 00:42:42.724149 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.724174 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:45939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 00:42:42.724196 8221 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 00:42:42.724496 7283 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb -> Peer 406dea23b8464ffd88e6881c301c3094 (127.30.104.252:45939): Couldn't send request to peer 406dea23b8464ffd88e6881c301c3094. Status: Network error: Client connection negotiation failed: client connection to 127.30.104.252:45939: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 00:42:42.724869 8095 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 82e929c186674536a7841a1dc3b681b5 (127.30.104.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.725868 8095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.725947 8095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:42.726164 8221 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 00:42:42.726246 8221 log.cc:1196] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 00:42:42.726372 8093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.726475 8221 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Deleting consensus metadata
I20260809 00:42:42.726480 8093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:42.726606 8221 master_runner.cc:315] Copying system tablet from 127.30.104.254:40589
I20260809 00:42:42.726900 8158 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 82e929c186674536a7841a1dc3b681b5 (127.30.104.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.727274 8221 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: tablet copy: Beginning tablet copy session from remote peer at address 127.30.104.254:40589
I20260809 00:42:42.728224 8359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.728412 8359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:42.735005 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.728744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.30.104.254:40589)
0809 00:42:42.728788 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.728790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.728945 (+ 155us) server_negotiation.cc:407] Connection header received
0809 00:42:42.729024 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.729024 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.729036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.729089 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.729432 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.729958 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.730702 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.730974 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.731046 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.732112 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.732114 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.732114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.732122 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.733664 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.734046 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.734048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.734048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.734113 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.734437 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.734438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.734439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.734576 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.734654 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.734807 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.734899 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:42.735034 8371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.728700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42470)
0809 00:42:42.728869 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.728878 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.728969 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.729110 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.729111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.729298 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.729345 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.729348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.730061 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.730063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.730609 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.730611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.731055 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.731057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.731124 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.731672 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.731677 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.732013 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.733775 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.733776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.733777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.733971 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.734197 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.734197 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.734198 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.734359 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.734645 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.734646 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.734696 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.734780 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.734898 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":95,"threads_started":1}
I20260809 00:42:42.735486 7330 tablet_copy_service.cc:140] P 1f8a3704ece84f08a8e873f37aba8dcb: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 406dea23b8464ffd88e6881c301c3094 ({username='oryx', principal='oryx/127.30.104.252@KRBTEST.COM'} at 127.0.0.1:42470)
I20260809 00:42:42.735549 7330 tablet_copy_service.cc:161] P 1f8a3704ece84f08a8e873f37aba8dcb: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 406dea23b8464ffd88e6881c301c3094 at {username='oryx', principal='oryx/127.30.104.252@KRBTEST.COM'} at 127.0.0.1:42470: session id = 406dea23b8464ffd88e6881c301c3094-00000000000000000000000000000000
I20260809 00:42:42.736042 7330 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 00:42:42.736604 8221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 00:42:42.737672 8221 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: tablet copy: Starting download of 0 data blocks...
I20260809 00:42:42.737860 8221 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: tablet copy: Starting download of 1 WAL segments...
I20260809 00:42:42.738627 8221 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 00:42:42.739737 7330 tablet_copy_service.cc:342] P 1f8a3704ece84f08a8e873f37aba8dcb: Request end of tablet copy session 406dea23b8464ffd88e6881c301c3094-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.104.252@KRBTEST.COM'} at 127.0.0.1:42470
I20260809 00:42:42.739822 7330 tablet_copy_service.cc:433] P 1f8a3704ece84f08a8e873f37aba8dcb: ending tablet copy session 406dea23b8464ffd88e6881c301c3094-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 406dea23b8464ffd88e6881c301c3094
I20260809 00:42:42.742455 8221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:42.744001 8373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.744024 8372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:42.748811 8221 server_base.cc:1034] running on GCE node
W20260809 00:42:42.748811 8375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:42.749075 8221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:42.749315 8221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:42.750468 8221 hybrid_clock.cc:648] HybridClock initialized: now 1786236162750443 us; error 46 us; skew 500 ppm
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:42.752115 8221 init.cc:377] Logged in from keytab as oryx/127.30.104.252@KRBTEST.COM (short username oryx)
I20260809 00:42:42.752440 8221 webserver.cc:460] Webserver started at http://127.30.104.252:45357/ using document root <none> and password file <none>
I20260809 00:42:42.752578 8221 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:42.752614 8221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:42.753300 8221 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:42.753916 8384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.754050 8221 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.754091 8221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
uuid: "406dea23b8464ffd88e6881c301c3094"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c0c0ca6e9085177fc26eb1d0a2727ec3"
server_key_iv: "587a649485d480dc54bae3912d50a3c2"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.754156 8221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:42.765933 8221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:42.767899 8221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:42.771203 8221 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.252:45939
I20260809 00:42:42.771220 8462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.252:45939 every 8 connection(s)
I20260809 00:42:42.771588 8221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
I20260809 00:42:42.776110 8463 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:42:42.776580 8463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Bootstrap starting.
I20260809 00:42:42.778971 8463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:42:42.779213 8463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: Bootstrap complete.
I20260809 00:42:42.779492 8463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } }
I20260809 00:42:42.779600 8463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 406dea23b8464ffd88e6881c301c3094, State: Initialized, Role: LEARNER
I20260809 00:42:42.779650 8463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } }
I20260809 00:42:42.779837 8463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:42.779831 8466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:42.780082 8466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: This master's current role is: LEARNER
I20260809 00:42:42.781407 8482 catalog_manager.cc:1259] Loaded cluster ID: 36fc3005ae8f4261a5790d5d99715fd0
I20260809 00:42:42.781499 8482 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: loading cluster ID for follower catalog manager: success
I20260809 00:42:42.782989 8482 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: acquiring CA information for follower catalog manager: success
I20260809 00:42:42.783319 8482 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 00:42:42.844734 8355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:42.845089 8355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:42.845134 8355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:42.846949 8355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:42.847028 8355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:42.847074 8355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:42.849432 8355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab
--principal=oryx/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:33821
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=39807
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041,127.30.104.252:45939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8355
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:42.849737 8355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:42.850047 8355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:42.855611 8498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.855634 8492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:42.855634 8493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:42.856434 8355 server_base.cc:1034] running on GCE node
I20260809 00:42:42.856700 8355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:42.856992 8355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:42.858181 8355 hybrid_clock.cc:648] HybridClock initialized: now 1786236162858168 us; error 30 us; skew 500 ppm
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:42.861266 8355 init.cc:377] Logged in from keytab as oryx/127.30.104.193@KRBTEST.COM (short username oryx)
I20260809 00:42:42.861871 8355 webserver.cc:460] Webserver started at http://127.30.104.193:39807/ using document root <none> and password file <none>
I20260809 00:42:42.862143 8355 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:42.862234 8355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:42.863880 8355 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:42:42.864621 8506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.864861 8355 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:42.864965 8355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "fb53d311bae2400096cd1463ff086f92"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "f719885e452c47f86e0d70c1ca4f1544"
server_key_iv: "237c07faa7174c01be00fc7188acb524"
server_key_version: "encryptionkey@0"
I20260809 00:42:42.865438 8355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:42.879710 7772 heartbeater.cc:503] Master 127.30.104.254:40589 was elected leader, sending a full tablet report...
I20260809 00:42:42.891196 8355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:42.893354 8355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:42.893527 8355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:42.893798 8355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:42.894142 8355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:42.894210 8355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.894254 8355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:42.894274 8355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:42.900856 8355 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:33821
I20260809 00:42:42.900919 8676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:33821 every 8 connection(s)
I20260809 00:42:42.901314 8355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:42.904703 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 8355
I20260809 00:42:42.904881 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 7589
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.253@KRBTEST.COM
I20260809 00:42:42.920872 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.902436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:49231 (local address 127.30.104.253:37041)
0809 00:42:42.902505 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.902507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.902810 (+ 303us) server_negotiation.cc:407] Connection header received
0809 00:42:42.903826 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.903828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.903841 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.903899 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.911378 (+ 7479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.912079 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.913746 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.914061 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.914102 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.917688 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.917690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.917690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.917699 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.919170 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.919797 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.919799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.919799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.919859 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.920121 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.920122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.920122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.920265 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.920311 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.920650 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.920736 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:42.921751 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.902164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.30.104.193:49231)
0809 00:42:42.902685 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.902698 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.903724 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.904010 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.904015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.904462 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.904858 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.904862 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.912234 (+ 7372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.912235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.913641 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.913645 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.914225 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.914226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.914408 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.915676 (+ 1268us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.915688 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.917561 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.919291 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.919300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.919303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.919708 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.919917 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.919918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.919918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.920060 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.920342 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.920348 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.920516 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.921258 (+ 742us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.921456 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":13,"thread_start_us":111,"threads_started":1}
I20260809 00:42:42.922509 8682 heartbeater.cc:347] Connected to a master server at 127.30.104.253:37041
I20260809 00:42:42.922602 8682 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:42.922783 8682 heartbeater.cc:511] Master 127.30.104.253:37041 requested a full tablet report, sending...
I20260809 00:42:42.923323 8146 ts_manager.cc:194] Registered new tserver with Master: fb53d311bae2400096cd1463ff086f92 (127.30.104.193:33821)
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
Aug 09 00:42:42 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236162, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.252@KRBTEST.COM
I20260809 00:42:42.929486 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:44235
--local_ip_for_outbound_sockets=127.30.104.194
--tserver_master_addrs=127.30.104.254:40589
--webserver_port=42505
--webserver_interface=127.30.104.194
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab
--principal=oryx/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041,127.30.104.252:45939 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:42:42.931202 8685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.902514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:51951)
0809 00:42:42.903549 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.903560 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.903719 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.904048 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.904049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.904481 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.904807 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.904813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.923969 (+ 19156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.923969 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.924532 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.924533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.925128 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.925130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.925226 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.925776 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.925782 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.927425 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.929667 (+ 2242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.929668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.929669 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.929904 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.930136 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.930137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.930137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.930201 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.930808 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.930809 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.930864 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.930942 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.931053 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"mutex_wait_us":14,"thread_start_us":73,"threads_started":1}
I20260809 00:42:42.931244 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.902727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:51951 (local address 127.30.104.254:40589)
0809 00:42:42.902790 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.902792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.903630 (+ 838us) server_negotiation.cc:407] Connection header received
0809 00:42:42.903958 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.903959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.903972 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.904028 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.904931 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.905616 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.924645 (+ 19029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.925021 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.925097 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.927837 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.927838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.927839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.927847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.929569 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.929976 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.929977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.929977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.930057 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.930297 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.930298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.930298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.930722 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.930798 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.931008 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.931111 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:42:42.931881 8680 heartbeater.cc:347] Connected to a master server at 127.30.104.254:40589
I20260809 00:42:42.931957 8680 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:42.932044 8680 heartbeater.cc:511] Master 127.30.104.254:40589 requested a full tablet report, sending...
I20260809 00:42:42.932422 7305 ts_manager.cc:194] Re-registered known tserver with Master: fb53d311bae2400096cd1463ff086f92 (127.30.104.193:33821)
I20260809 00:42:42.932799 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.902153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:33169 (local address 127.30.104.252:45939)
0809 00:42:42.902308 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:42.902311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:42.903529 (+ 1218us) server_negotiation.cc:407] Connection header received
0809 00:42:42.904094 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:42.904096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:42.904111 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:42.904174 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:42.925153 (+ 20979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.925988 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.926831 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.927194 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.927272 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.929603 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:42.929605 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:42.929606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:42.929614 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:42.931370 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.931821 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.931823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.931823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.931895 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:42.932134 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:42.932135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:42.932135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:42.932305 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:42.932364 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:42.932603 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 00:42:42.932697 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
I20260809 00:42:42.932932 7305 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:51951
I20260809 00:42:42.933331 8686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:42.902165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.30.104.193:33169)
0809 00:42:42.902351 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:42.902397 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:42.903711 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:42.904193 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:42.904193 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:42.904475 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:42.904807 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.904813 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.926087 (+ 21274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.926088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.926722 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:42.926725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:42.927337 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:42.927338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:42.927418 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:42.927866 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:42.927872 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:42.929491 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:42.931500 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.931501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.931502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.931725 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.931989 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:42.931990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:42.931990 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:42.932051 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:42.932407 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:42.932408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:42.932474 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 00:42:42.932880 (+ 406us) client_negotiation.cc:240] Negotiation successful
0809 00:42:42.933001 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":43,"spinlock_wait_cycles":384,"thread_start_us":108,"threads_started":1}
I20260809 00:42:42.933750 8678 heartbeater.cc:347] Connected to a master server at 127.30.104.252:45939
I20260809 00:42:42.933810 8678 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:42.933892 8678 heartbeater.cc:511] Master 127.30.104.252:45939 requested a full tablet report, sending...
I20260809 00:42:42.934265 8414 ts_manager.cc:194] Registered new tserver with Master: fb53d311bae2400096cd1463ff086f92 (127.30.104.193:33821)
W20260809 00:42:43.017477 8689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:43.017690 8689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:43.017715 8689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:43.018926 8689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:43.018975 8689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:43.019002 8689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:43.020612 8689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab
--principal=oryx/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:44235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=42505
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041,127.30.104.252:45939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:43.020884 8689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:43.021129 8689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:43.026710 8695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:43.026721 8697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:43.026816 8694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:43.026961 8689 server_base.cc:1034] running on GCE node
I20260809 00:42:43.027145 8689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:43.027354 8689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:43.028540 8689 hybrid_clock.cc:648] HybridClock initialized: now 1786236163028534 us; error 65 us; skew 500 ppm
Aug 09 00:42:43 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236163, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:43.031217 8689 init.cc:377] Logged in from keytab as oryx/127.30.104.194@KRBTEST.COM (short username oryx)
I20260809 00:42:43.031728 8689 webserver.cc:460] Webserver started at http://127.30.104.194:42505/ using document root <none> and password file <none>
I20260809 00:42:43.031955 8689 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:43.032013 8689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:43.033347 8689 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:43.033973 8712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:43.034257 8689 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:43.034353 8689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "b6da8911a57e4f19958a580c25f21555"
format_stamp: "Formatted at 2026-08-09 00:42:41 on dist-test-slave-c12x"
server_key: "07eaad8685325a813e312118f1f9af32"
server_key_iv: "fb14dbaa403c6e204b47430cae1ea561"
server_key_version: "encryptionkey@0"
I20260809 00:42:43.034731 8689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:43.054015 8689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:43.056583 8689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:43.056712 8689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:43.056977 8689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:43.057289 8689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:43.057322 8689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:43.057344 8689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:43.057358 8689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:43.063598 8689 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:44235
I20260809 00:42:43.063725 8873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:44235 every 8 connection(s)
I20260809 00:42:43.064083 8689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:43.067224 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 8689
I20260809 00:42:43.067425 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 7779
Aug 09 00:42:43 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236163, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.252@KRBTEST.COM
I20260809 00:42:43.088083 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.065445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:44373 (local address 127.30.104.252:45939)
0809 00:42:43.065514 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.065516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.065584 (+ 68us) server_negotiation.cc:407] Connection header received
0809 00:42:43.066536 (+ 952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.066538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.066551 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.066585 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.077686 (+ 11101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.078386 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.079973 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.080280 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.080321 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.083788 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.083791 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.083791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.083800 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.086243 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.086764 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.086765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.086766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.086821 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.087064 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.087065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.087065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.087377 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.087468 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.087834 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.087938 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:42:43.088958 8883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.065266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.30.104.194:44373)
0809 00:42:43.065533 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.065543 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.066389 (+ 846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.066682 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.066683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.067209 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:43.077520 (+ 10311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.077529 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.078559 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.078560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.079855 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.079859 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.080484 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.080485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.080645 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.081614 (+ 969us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:43.081625 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:43.083665 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:43.086340 (+ 2675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.086350 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.086352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.086682 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.086878 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.086878 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.086879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.087004 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.087503 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:43.087511 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:43.087696 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.088471 (+ 775us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.088683 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":59,"thread_start_us":64,"threads_started":1}
I20260809 00:42:43.089706 8876 heartbeater.cc:347] Connected to a master server at 127.30.104.252:45939
I20260809 00:42:43.089802 8876 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:43.089991 8876 heartbeater.cc:511] Master 127.30.104.252:45939 requested a full tablet report, sending...
I20260809 00:42:43.090482 8414 ts_manager.cc:194] Registered new tserver with Master: b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235)
Aug 09 00:42:43 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236163, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:43.100414 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.065301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:57611 (local address 127.30.104.254:40589)
0809 00:42:43.065345 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.065348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.065536 (+ 188us) server_negotiation.cc:407] Connection header received
0809 00:42:43.066507 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.066509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.066523 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.066558 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.093274 (+ 26716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.093816 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.094652 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.094912 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.095000 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.097351 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.097354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.097354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.097382 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.099016 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.099381 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.099382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.099383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.099531 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.099749 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.099749 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.099750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.099903 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.099948 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.100165 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.100263 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:43.101151 8882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.064943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:57611)
0809 00:42:43.065411 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.065422 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.066395 (+ 973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.066683 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.066683 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.067209 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:43.067481 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.067489 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.093927 (+ 26438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.093928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.094559 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.094562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.095013 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.095014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.095094 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.095816 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:43.095822 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:43.097239 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:43.099107 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.099108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.099108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.099301 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.099611 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.099612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.099612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.099681 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.099998 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:43.099999 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:43.100049 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.100788 (+ 739us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.100970 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":77,"thread_start_us":105,"threads_started":1}
Aug 09 00:42:43 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236163, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.253@KRBTEST.COM
I20260809 00:42:43.104228 8877 heartbeater.cc:347] Connected to a master server at 127.30.104.254:40589
I20260809 00:42:43.104306 8877 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:43.104403 8877 heartbeater.cc:511] Master 127.30.104.254:40589 requested a full tablet report, sending...
I20260809 00:42:43.104739 7305 ts_manager.cc:194] Re-registered known tserver with Master: b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235)
I20260809 00:42:43.105072 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:33781
--local_ip_for_outbound_sockets=127.30.104.195
--tserver_master_addrs=127.30.104.254:40589
--webserver_port=34303
--webserver_interface=127.30.104.195
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab
--principal=oryx/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041,127.30.104.252:45939 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:42:43.105134 7305 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:57611
I20260809 00:42:43.108600 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.064953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:51391 (local address 127.30.104.253:37041)
0809 00:42:43.065003 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.065005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.065301 (+ 296us) server_negotiation.cc:407] Connection header received
0809 00:42:43.066494 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.066496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.066510 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.066566 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.100022 (+ 33456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.100625 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.101383 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.101677 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.101756 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.103733 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.103735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.103736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.103745 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.107254 (+ 3509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.107741 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.107742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.107743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.107816 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.108033 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.108034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.108034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.108161 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.108234 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.108402 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.108491 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:42:43.109016 8881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.064943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.30.104.194:51391)
0809 00:42:43.065142 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.065180 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.066361 (+ 1181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.066655 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.066666 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.067089 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:43.067482 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.067490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.100718 (+ 33228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.100719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.101262 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.101264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.101755 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.101756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.101830 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.102295 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:43.102300 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:43.103619 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:43.107351 (+ 3732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.107352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.107353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.107653 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.107886 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.107886 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.107887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.107961 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.108236 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:43.108237 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:43.108282 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.108359 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.108494 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
I20260809 00:42:43.109647 8878 heartbeater.cc:347] Connected to a master server at 127.30.104.253:37041
I20260809 00:42:43.109720 8878 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:43.109843 8878 heartbeater.cc:511] Master 127.30.104.253:37041 requested a full tablet report, sending...
I20260809 00:42:43.110263 8146 ts_manager.cc:194] Registered new tserver with Master: b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235)
I20260809 00:42:43.139053 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.136115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.0.0.1:56762)
0809 00:42:43.136184 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.136197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.136268 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.136456 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.136457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.136459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:43.136517 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.136519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.137223 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.137224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.138062 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.138065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.138793 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.138794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.138859 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.138863 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.138920 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.138967 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 00:42:43.139051 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.136221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56762 (local address 127.30.104.252:45939)
0809 00:42:43.136299 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.136301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.136312 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:43.136351 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.136351 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.136373 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.136450 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:43.136610 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.137130 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.138157 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.138698 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.138782 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.138859 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.138928 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.138965 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:42:43.139477 8432 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 00:42:43.139675 8466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f8a3704ece84f08a8e873f37aba8dcb. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:43.139804 8466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: This master's current role is: LEARNER
W20260809 00:42:43.198612 8884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:43.198851 8884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:43.198879 8884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:43.200198 8884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:43.200248 8884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:43.200278 8884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:43.202025 8884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab
--principal=oryx/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:33781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=34303
--tserver_master_addrs=127.30.104.254:40589,127.30.104.253:37041,127.30.104.252:45939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:43.202319 8884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:43.202591 8884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:43.208302 8891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:43.208387 8884 server_base.cc:1034] running on GCE node
W20260809 00:42:43.208280 8894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:43.208554 8890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:43.208803 8884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:43.209123 8884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:43.210336 8884 hybrid_clock.cc:648] HybridClock initialized: now 1786236163210325 us; error 37 us; skew 500 ppm
Aug 09 00:42:43 dist-test-slave-c12x krb5kdc[7240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236163, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:43.213677 8884 init.cc:377] Logged in from keytab as oryx/127.30.104.195@KRBTEST.COM (short username oryx)
I20260809 00:42:43.214306 8884 webserver.cc:460] Webserver started at http://127.30.104.195:34303/ using document root <none> and password file <none>
I20260809 00:42:43.214581 8884 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:43.214656 8884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:43.216259 8884 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:42:43.216979 8902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:43.217183 8884 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:43.217293 8884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "24133fd916f249f291f167baf4e2974a"
format_stamp: "Formatted at 2026-08-09 00:42:42 on dist-test-slave-c12x"
server_key: "c91502858866365013a3614c9fb678a1"
server_key_iv: "6524d6de3d6d2c86c5d29b99dbc20f0d"
server_key_version: "encryptionkey@0"
I20260809 00:42:43.217694 8884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:43.238991 8884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:43.241506 8884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:43.241636 8884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:43.241926 8884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:43.242300 8884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:43.242338 8884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:43.242408 8884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:43.242429 8884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:43.249248 8884 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:33781
I20260809 00:42:43.249332 9054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:33781 every 8 connection(s)
I20260809 00:42:43.249840 8884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:43.252106 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 8884
Aug 09 00:42:43 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236163, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
Aug 09 00:42:43 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236163, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.253@KRBTEST.COM
Aug 09 00:42:43 dist-test-slave-c12x krb5kdc[7240](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236163, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.252@KRBTEST.COM
I20260809 00:42:43.266430 7388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.251219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:55451 (local address 127.30.104.254:40589)
0809 00:42:43.251279 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.251282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.251642 (+ 360us) server_negotiation.cc:407] Connection header received
0809 00:42:43.252772 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.252774 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.252789 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.252826 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.254169 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.255117 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.257429 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.257755 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.257799 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.262380 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.262382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.262383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.262393 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.264836 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.265283 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.265284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.265285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.265348 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.265573 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.265574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.265575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.265725 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.265778 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.266220 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.266309 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:42:43.266666 9061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.250791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.30.104.195:58643)
0809 00:42:43.251002 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.251055 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.252624 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.253042 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.253043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.253565 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:43.253996 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.253997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.255073 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.255074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.256726 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.256728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.257272 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.257273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.257422 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.258164 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:43.258175 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:43.261120 (+ 2945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:43.264679 (+ 3559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.264680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.264680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.264897 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.265626 (+ 729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.265627 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.265628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.265697 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.266010 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:43.266011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:43.266059 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.266159 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.266366 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":349,"thread_start_us":107,"threads_started":1}
I20260809 00:42:43.266870 9062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.251149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:55451)
0809 00:42:43.251567 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.251577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.252624 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.253042 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.253043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.253576 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:43.253928 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.253937 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.255280 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.255280 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.257329 (+ 2049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.257330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.257885 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.257886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.257961 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.258471 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:43.258476 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:43.262118 (+ 3642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:43.264937 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.264938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.264939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.265184 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.265425 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.265426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.265426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.265494 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.265814 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:43.265823 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:43.266040 (+ 217us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.266594 (+ 554us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.266718 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":107,"thread_start_us":78,"threads_started":1}
I20260809 00:42:43.267186 8217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.250897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:58643 (local address 127.30.104.253:37041)
0809 00:42:43.250943 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.250945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.251155 (+ 210us) server_negotiation.cc:407] Connection header received
0809 00:42:43.252773 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.252774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.252789 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.252826 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.254101 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.254886 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.256817 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.257141 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.257201 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.261229 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.261231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.261231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.261240 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.264602 (+ 3362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.265471 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.265473 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.265473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.265543 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.265766 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.265767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.265767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.265906 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.265979 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.266977 (+ 998us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.267065 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:43.268371 9058 heartbeater.cc:347] Connected to a master server at 127.30.104.253:37041
I20260809 00:42:43.268433 9071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.253805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47216 (local address 127.30.104.253:37041)
0809 00:42:43.255469 (+ 1664us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.255471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.257794 (+ 2323us) server_negotiation.cc:407] Connection header received
0809 00:42:43.257853 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.257854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.257868 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.257951 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.259595 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.260333 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.261525 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.261853 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.261893 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.263427 (+ 1534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.263428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.263429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.263468 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.266092 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.267376 (+ 1284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.267378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.267378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.267492 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.267745 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.267747 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.267747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.267894 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.267972 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.268222 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.268330 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1617,"thread_start_us":65,"threads_started":1}
I20260809 00:42:43.268494 9058 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:43.268532 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.250838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:42055 (local address 127.30.104.252:45939)
0809 00:42:43.250896 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.250898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.251155 (+ 257us) server_negotiation.cc:407] Connection header received
0809 00:42:43.252762 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.252765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.252781 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.252818 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.254101 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.255121 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.256784 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.257227 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.257327 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.261510 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.261512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.261513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.261522 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.263427 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.264342 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.264343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.264344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.264409 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.267898 (+ 3489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.267900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.267900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.268053 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.268108 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.268332 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.268430 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:42:43.268724 9058 heartbeater.cc:511] Master 127.30.104.253:37041 requested a full tablet report, sending...
I20260809 00:42:43.269052 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.250791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.30.104.195:42055)
0809 00:42:43.251001 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.251055 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.252608 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.253035 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.253040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.253562 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:43.253989 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.253992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.255263 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.255264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.256689 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.256692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.257389 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.257391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.257479 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.258940 (+ 1461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:43.258946 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:43.261381 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:43.263892 (+ 2511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.263902 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.263905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.264265 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.264480 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:43.264481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:43.264482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:43.264628 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:43.268145 (+ 3517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:43.268146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:43.268191 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.268778 (+ 587us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.268927 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260809 00:42:43.269083 9056 heartbeater.cc:347] Connected to a master server at 127.30.104.254:40589
I20260809 00:42:43.269155 9056 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:43.269207 8146 ts_manager.cc:194] Registered new tserver with Master: 24133fd916f249f291f167baf4e2974a (127.30.104.195:33781)
I20260809 00:42:43.269227 9056 heartbeater.cc:511] Master 127.30.104.254:40589 requested a full tablet report, sending...
I20260809 00:42:43.269551 7305 ts_manager.cc:194] Re-registered known tserver with Master: 24133fd916f249f291f167baf4e2974a (127.30.104.195:33781)
I20260809 00:42:43.269596 9055 heartbeater.cc:347] Connected to a master server at 127.30.104.252:45939
I20260809 00:42:43.269644 9055 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:43.269730 9055 heartbeater.cc:511] Master 127.30.104.252:45939 requested a full tablet report, sending...
I20260809 00:42:43.270051 7305 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:55451
I20260809 00:42:43.270059 8414 ts_manager.cc:194] Registered new tserver with Master: 24133fd916f249f291f167baf4e2974a (127.30.104.195:33781)
I20260809 00:42:43.270728 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.256580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56774 (local address 127.30.104.252:45939)
0809 00:42:43.258546 (+ 1966us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.258548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.258558 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:43.258584 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.258585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.258597 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.258680 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.259480 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.260357 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.264346 (+ 3989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.264676 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.264741 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.267087 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.267089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.267090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.267099 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.269357 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.269772 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.269773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.269774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.269840 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.270159 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.270160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.270160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.270321 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.270410 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.270542 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.270627 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1938,"thread_start_us":75,"threads_started":1}
I20260809 00:42:43.273164 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.253951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42472 (local address 127.30.104.254:40589)
0809 00:42:43.255537 (+ 1586us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.255540 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.257941 (+ 2401us) server_negotiation.cc:407] Connection header received
0809 00:42:43.258620 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.258621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.258637 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.258693 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.259924 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.260752 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.263308 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.263693 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.263740 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.265956 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.265958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.265958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.265964 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.271949 (+ 5985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.272330 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.272331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.272331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.272404 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.272620 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.272621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.272621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.272743 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.272860 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.272961 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.273057 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1556,"thread_start_us":76,"threads_started":1}
I20260809 00:42:43.274780 7305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42472:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:43.276481 7305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:43.277276 8354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: NON_VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: true } } }
I20260809 00:42:43.277386 8354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:43.278831 9079 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: attempting to promote NON_VOTER 406dea23b8464ffd88e6881c301c3094 to VOTER
I20260809 00:42:43.278970 9079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } }
I20260809 00:42:43.279940 8432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 LEARNER]: Refusing update from remote peer 1f8a3704ece84f08a8e873f37aba8dcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 00:42:43.280288 8158 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Refusing update from remote peer 1f8a3704ece84f08a8e873f37aba8dcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 00:42:43.280309 8354 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 00:42:43.280483 8354 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 00:42:43.281021 9078 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 406dea23b8464ffd88e6881c301c3094 (127.30.104.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:43.281551 8158 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 406dea23b8464ffd88e6881c301c3094 (127.30.104.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:43.282274 9078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:43.282373 9078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:43.282339 9079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:43.282439 8354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:43.282524 8354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:43.282449 9079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:43.282765 8432 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 406dea23b8464ffd88e6881c301c3094 (127.30.104.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:43.283618 9096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:43.284015 8466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:43.284104 8466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:43.283951 9096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:43.289281 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.283566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33821 (local address 127.0.0.1:55324)
0809 00:42:43.283898 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.283908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.283971 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.284715 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.284716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.284718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:43.284778 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.284780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.286417 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.286418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.287868 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.287872 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.289015 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.289016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.289089 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.289109 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.289142 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.289187 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":114,"threads_started":1}
I20260809 00:42:43.289453 9097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.283383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44235 (local address 127.0.0.1:50576)
0809 00:42:43.283729 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.283742 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.283799 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.285069 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.285070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.285072 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:43.285116 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.285118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.287537 (+ 2419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.287539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.288488 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.288491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.289201 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.289202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.289266 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.289270 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.289314 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.289356 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319}
I20260809 00:42:43.289619 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.284081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50576 (local address 127.30.104.194:44235)
0809 00:42:43.284844 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.284849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.284859 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:43.284896 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.284899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.284916 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.284994 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:43.285219 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.286337 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.288578 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.289088 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.289226 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.289296 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.289328 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.289365 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":100,"threads_started":1}
I20260809 00:42:43.290181 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.283688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55324 (local address 127.30.104.193:33821)
0809 00:42:43.284534 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.284542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.284552 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:43.284600 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.284602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.284620 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.284709 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:43.284869 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.286322 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.288004 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.288891 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.289931 (+ 1040us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.290017 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.290047 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.290095 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":95,"threads_started":1}
I20260809 00:42:43.290504 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.283790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:33781 (local address 127.0.0.1:41946)
0809 00:42:43.284294 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.284305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.284361 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.285854 (+ 1493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.285855 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.285857 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:43.285901 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.285903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.288331 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.288332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.289534 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.289538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.290251 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.290252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.290315 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.290320 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.290377 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.290420 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":73,"threads_started":1}
I20260809 00:42:43.290438 8774 tablet_service.cc:1467] Processing CreateTablet for tablet ef448d66ebca474699a1824d05748e97 (DEFAULT_TABLE table=test-table [id=f5458cb3e13147d48e782d05997958fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:43.290740 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.284457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.30.104.195:33781)
0809 00:42:43.285664 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.285669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.285681 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:42:43.285725 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.285727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.285744 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.285858 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:43.285968 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.288230 (+ 2262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.289633 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.290147 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.290251 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.290305 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.290594 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.290651 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1165,"thread_start_us":107,"threads_started":1}
I20260809 00:42:43.290920 8774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef448d66ebca474699a1824d05748e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:43.291425 8965 tablet_service.cc:1467] Processing CreateTablet for tablet ef448d66ebca474699a1824d05748e97 (DEFAULT_TABLE table=test-table [id=f5458cb3e13147d48e782d05997958fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:43.291895 8965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef448d66ebca474699a1824d05748e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:43.295941 9104 tablet_bootstrap.cc:492] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Bootstrap starting.
I20260809 00:42:43.295964 8574 tablet_service.cc:1467] Processing CreateTablet for tablet ef448d66ebca474699a1824d05748e97 (DEFAULT_TABLE table=test-table [id=f5458cb3e13147d48e782d05997958fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:43.296368 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef448d66ebca474699a1824d05748e97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:43.296627 9104 tablet_bootstrap.cc:654] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:43.296859 9104 log.cc:824] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:43.297657 9104 tablet_bootstrap.cc:492] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: No bootstrap required, opened a new log
I20260809 00:42:43.297708 9104 ts_tablet_manager.cc:1397] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:43.299201 9104 raft_consensus.cc:348] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.299373 9104 raft_consensus.cc:374] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:43.299414 9104 raft_consensus.cc:729] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24133fd916f249f291f167baf4e2974a, State: Initialized, Role: FOLLOWER
I20260809 00:42:43.299628 9104 consensus_queue.cc:260] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.299770 9103 tablet_bootstrap.cc:492] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Bootstrap starting.
I20260809 00:42:43.299821 9108 tablet_bootstrap.cc:492] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Bootstrap starting.
I20260809 00:42:43.299925 9104 ts_tablet_manager.cc:1428] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:43.300014 9056 heartbeater.cc:503] Master 127.30.104.254:40589 was elected leader, sending a full tablet report...
I20260809 00:42:43.300616 9103 tablet_bootstrap.cc:654] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:43.301040 9103 log.cc:824] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:43.301839 9108 tablet_bootstrap.cc:654] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:43.301934 9103 tablet_bootstrap.cc:492] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: No bootstrap required, opened a new log
I20260809 00:42:43.302032 9103 ts_tablet_manager.cc:1397] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:42:43.302189 9108 log.cc:824] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:43.303015 9108 tablet_bootstrap.cc:492] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: No bootstrap required, opened a new log
I20260809 00:42:43.303093 9108 ts_tablet_manager.cc:1397] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 00:42:43.304839 9103 raft_consensus.cc:348] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.305081 9103 raft_consensus.cc:374] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:43.305130 9103 raft_consensus.cc:729] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6da8911a57e4f19958a580c25f21555, State: Initialized, Role: FOLLOWER
I20260809 00:42:43.305282 9103 consensus_queue.cc:260] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.305580 9103 ts_tablet_manager.cc:1428] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 00:42:43.305739 8877 heartbeater.cc:503] Master 127.30.104.254:40589 was elected leader, sending a full tablet report...
I20260809 00:42:43.305914 9108 raft_consensus.cc:348] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.306181 9108 raft_consensus.cc:374] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:43.306224 9108 raft_consensus.cc:729] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb53d311bae2400096cd1463ff086f92, State: Initialized, Role: FOLLOWER
I20260809 00:42:43.306452 9108 consensus_queue.cc:260] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.306712 9108 ts_tablet_manager.cc:1428] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 00:42:43.306778 8680 heartbeater.cc:503] Master 127.30.104.254:40589 was elected leader, sending a full tablet report...
W20260809 00:42:43.314975 8880 tablet.cc:2369] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:42:43.402382 8683 tablet.cc:2369] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:42:43.501089 9059 tablet.cc:2369] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:42:43.572594 9113 raft_consensus.cc:482] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:43.572786 9113 raft_consensus.cc:504] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.573195 9113 leader_election.cc:288] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235), 24133fd916f249f291f167baf4e2974a (127.30.104.195:33781)
I20260809 00:42:43.576289 9102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.573685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:60759 (local address 127.30.104.195:33781)
0809 00:42:43.573741 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.573744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.573754 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:43.573785 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.573786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.573796 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.573869 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:43.574013 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.574485 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.575561 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.575926 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.576017 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.576026 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.576176 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.576216 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:42:43.576303 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.573582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:33781 (local address 127.30.104.193:60759)
0809 00:42:43.573665 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.573676 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.573731 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.573873 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.573874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.573876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:43.573927 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.573928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.574572 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.574574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.575463 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.575466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.576017 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.576018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.576085 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.576089 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.576172 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.576216 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":18,"thread_start_us":45,"threads_started":1}
I20260809 00:42:43.576349 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.573497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:40161 (local address 127.30.104.194:44235)
0809 00:42:43.573582 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.573584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.573605 (+ 21us) server_negotiation.cc:407] Connection header received
0809 00:42:43.573747 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.573749 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.573766 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.573835 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:43.573993 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.574489 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.575601 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.575976 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.576064 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.576072 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.576235 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.576274 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 00:42:43.576434 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.573381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44235 (local address 127.30.104.193:40161)
0809 00:42:43.573534 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:43.573547 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:43.573621 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:43.573844 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:43.573845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:43.573847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:43.573905 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.573908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.574575 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.574577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.575498 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:43.575501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.576077 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.576079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:43.576151 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.576157 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:42:43.576228 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 00:42:43.576271 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":40,"thread_start_us":58,"threads_started":1}
I20260809 00:42:43.576810 8998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef448d66ebca474699a1824d05748e97" candidate_uuid: "fb53d311bae2400096cd1463ff086f92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24133fd916f249f291f167baf4e2974a" is_pre_election: true
I20260809 00:42:43.576810 8809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef448d66ebca474699a1824d05748e97" candidate_uuid: "fb53d311bae2400096cd1463ff086f92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6da8911a57e4f19958a580c25f21555" is_pre_election: true
I20260809 00:42:43.576987 8998 raft_consensus.cc:2454] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb53d311bae2400096cd1463ff086f92 in term 0.
I20260809 00:42:43.576987 8809 raft_consensus.cc:2454] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb53d311bae2400096cd1463ff086f92 in term 0.
I20260809 00:42:43.577306 8509 leader_election.cc:302] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24133fd916f249f291f167baf4e2974a, fb53d311bae2400096cd1463ff086f92; no voters:
I20260809 00:42:43.577461 9113 raft_consensus.cc:2781] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:43.577569 9113 raft_consensus.cc:482] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:43.577607 9113 raft_consensus.cc:3037] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:43.578421 9113 raft_consensus.cc:504] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.578588 9113 leader_election.cc:288] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [CANDIDATE]: Term 1 election: Requested vote from peers b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235), 24133fd916f249f291f167baf4e2974a (127.30.104.195:33781)
I20260809 00:42:43.578804 8809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef448d66ebca474699a1824d05748e97" candidate_uuid: "fb53d311bae2400096cd1463ff086f92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6da8911a57e4f19958a580c25f21555"
I20260809 00:42:43.578815 8998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef448d66ebca474699a1824d05748e97" candidate_uuid: "fb53d311bae2400096cd1463ff086f92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24133fd916f249f291f167baf4e2974a"
I20260809 00:42:43.578892 8809 raft_consensus.cc:3037] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:43.578924 8998 raft_consensus.cc:3037] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:43.579710 8809 raft_consensus.cc:2454] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb53d311bae2400096cd1463ff086f92 in term 1.
I20260809 00:42:43.579824 8998 raft_consensus.cc:2454] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb53d311bae2400096cd1463ff086f92 in term 1.
I20260809 00:42:43.579888 8509 leader_election.cc:302] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6da8911a57e4f19958a580c25f21555, fb53d311bae2400096cd1463ff086f92; no voters:
I20260809 00:42:43.580021 9113 raft_consensus.cc:2781] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:43.580183 9113 raft_consensus.cc:686] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 1 LEADER]: Becoming Leader. State: Replica: fb53d311bae2400096cd1463ff086f92, State: Running, Role: LEADER
I20260809 00:42:43.580312 9113 consensus_queue.cc:237] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } }
I20260809 00:42:43.581228 7305 catalog_manager.cc:5495] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb53d311bae2400096cd1463ff086f92 (127.30.104.193). New cstate: current_term: 1 leader_uuid: "fb53d311bae2400096cd1463ff086f92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:43.647550 9098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.645249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55336 (local address 127.30.104.193:33821)
0809 00:42:43.645333 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.645335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.645343 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:43.645381 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.645382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.645400 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.645469 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:43.645631 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.646114 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.646835 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.647018 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.647075 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.647209 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:43.647312 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:43.647376 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.647409 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.647476 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 00:42:43.649462 8809 raft_consensus.cc:1261] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 1 FOLLOWER]: Refusing update from remote peer fb53d311bae2400096cd1463ff086f92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:43.649492 8998 raft_consensus.cc:1261] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 1 FOLLOWER]: Refusing update from remote peer fb53d311bae2400096cd1463ff086f92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:43.649874 9113 consensus_queue.cc:1035] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:43.650022 9117 consensus_queue.cc:1035] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:43.653198 9123 mvcc.cc:205] Tried to move back new op lower bound from 7316423326305329152 to 7316423326025519104. Current Snapshot: MvccSnapshot[applied={T|T < 7316423326305329152}]
I20260809 00:42:43.655242 7303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:43.655361 7303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42472:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:43.656848 9124 mvcc.cc:205] Tried to move back new op lower bound from 7316423326305329152 to 7316423326025519104. Current Snapshot: MvccSnapshot[applied={T|T < 7316423326305329152}]
I20260809 00:42:43.657995 7303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1f8a3704ece84f08a8e873f37aba8dcb: Sending DeleteTablet for 3 replicas of tablet ef448d66ebca474699a1824d05748e97
I20260809 00:42:43.658305 8774 tablet_service.cc:1514] Processing DeleteTablet for tablet ef448d66ebca474699a1824d05748e97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:43 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:50576
I20260809 00:42:43.658409 8965 tablet_service.cc:1514] Processing DeleteTablet for tablet ef448d66ebca474699a1824d05748e97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:43 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:41946
I20260809 00:42:43.658617 9134 tablet_replica.cc:330] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: stopping tablet replica
I20260809 00:42:43.658705 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet ef448d66ebca474699a1824d05748e97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:43 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:55324
I20260809 00:42:43.658849 9134 raft_consensus.cc:2229] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:43.658948 9135 tablet_replica.cc:330] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: stopping tablet replica
I20260809 00:42:43.659050 9135 raft_consensus.cc:2229] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:43.659117 9135 raft_consensus.cc:2258] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:43.659267 9138 tablet_replica.cc:330] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: stopping tablet replica
I20260809 00:42:43.659353 9138 raft_consensus.cc:2229] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:43.659472 9138 raft_consensus.cc:2258] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:43.659072 9134 raft_consensus.cc:2258] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:43.659704 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 7258
I20260809 00:42:43.666824 9138 ts_tablet_manager.cc:1905] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:43.666879 9134 ts_tablet_manager.cc:1905] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:43.668814 9134 ts_tablet_manager.cc:1918] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:43.668947 9134 log.cc:1196] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/ef448d66ebca474699a1824d05748e97
I20260809 00:42:43.669224 9134 ts_tablet_manager.cc:1939] T ef448d66ebca474699a1824d05748e97 P b6da8911a57e4f19958a580c25f21555: Deleting consensus metadata
I20260809 00:42:43.669978 9138 ts_tablet_manager.cc:1918] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:43.670063 9138 log.cc:1196] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/ef448d66ebca474699a1824d05748e97
I20260809 00:42:43.670372 9138 ts_tablet_manager.cc:1939] T ef448d66ebca474699a1824d05748e97 P fb53d311bae2400096cd1463ff086f92: Deleting consensus metadata
I20260809 00:42:43.670497 9135 ts_tablet_manager.cc:1905] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:43.672173 9135 ts_tablet_manager.cc:1918] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:43.672322 9135 log.cc:1196] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/ef448d66ebca474699a1824d05748e97
I20260809 00:42:43.672708 9135 ts_tablet_manager.cc:1939] T ef448d66ebca474699a1824d05748e97 P 24133fd916f249f291f167baf4e2974a: Deleting consensus metadata
W20260809 00:42:43.675239 8905 connection.cc:535] server connection from 127.0.0.1:41946 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 00:42:43.675262 8509 connection.cc:535] server connection from 127.0.0.1:55324 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 00:42:43.675450 8717 connection.cc:535] server connection from 127.0.0.1:50576 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 00:42:43.681171 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.659228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56788 (local address 127.30.104.252:45939)
0809 00:42:43.659417 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.659419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.671831 (+ 12412us) server_negotiation.cc:407] Connection header received
0809 00:42:43.671963 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.671964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.671982 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.672018 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.672919 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.673503 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.674273 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.674540 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.674608 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.677381 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.677383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.677383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.677394 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.679395 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.679898 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.679900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.679900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.679967 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.680200 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.680201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.680202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.680330 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.680425 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.680883 (+ 458us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.681041 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260809 00:42:43.685994 8684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.677696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56804 (local address 127.30.104.252:45939)
0809 00:42:43.677742 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.677744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.678092 (+ 348us) server_negotiation.cc:407] Connection header received
0809 00:42:43.678192 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.678193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.678204 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.678235 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.678940 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.679771 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.680995 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.681330 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.681372 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.682871 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.682873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.682873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.682880 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.684561 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.685035 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.685036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.685037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.685097 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.685330 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.685330 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.685331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.685475 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.685544 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.685782 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.685866 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:42:43.686010 9071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:43.677529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47228 (local address 127.30.104.253:37041)
0809 00:42:43.677578 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:43.677580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:43.677992 (+ 412us) server_negotiation.cc:407] Connection header received
0809 00:42:43.678178 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:43.678180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:43.678197 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:43.678231 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:43.679042 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.679855 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.681039 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:43.681355 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:43.681398 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:43.682940 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:43.682942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:43.682942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:43.682950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:43.684720 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.685135 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.685136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.685136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.685199 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:43.685420 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:43.685421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:43.685421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:43.685563 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:43.685676 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:43.685799 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 00:42:43.685892 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:42:44.655565 9153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:44.655160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:41133)
0809 00:42:44.655350 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:44.655493 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":98,"threads_started":1}
W20260809 00:42:44.655855 8877 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:40589 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:44.658829 9154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:44.658602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:40921)
0809 00:42:44.658720 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:44.658789 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
W20260809 00:42:44.659049 9056 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:40589 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:44.659857 9155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:44.659628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:57475)
0809 00:42:44.659752 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:44.659821 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":55,"threads_started":1}
W20260809 00:42:44.660074 8680 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:40589 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:45.157315 9156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f8a3704ece84f08a8e873f37aba8dcb)
I20260809 00:42:45.157534 9156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } }
I20260809 00:42:45.157856 9156 leader_election.cc:288] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589), 406dea23b8464ffd88e6881c301c3094 (127.30.104.252:45939)
I20260809 00:42:45.158108 9157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1f8a3704ece84f08a8e873f37aba8dcb)
I20260809 00:42:45.158229 9157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } }
I20260809 00:42:45.158458 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.158054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42918)
0809 00:42:45.158346 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.158414 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":55,"threads_started":1}
I20260809 00:42:45.158491 9157 leader_election.cc:288] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589), 82e929c186674536a7841a1dc3b681b5 (127.30.104.253:37041)
W20260809 00:42:45.158737 8124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:45.158846 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.158548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42934)
0809 00:42:45.158743 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.158803 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":35,"threads_started":1}
W20260809 00:42:45.159013 8385 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:45.159363 9158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.159263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42950)
0809 00:42:45.159301 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.159331 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 00:42:45.159490 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.159358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42952)
0809 00:42:45.159416 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.159465 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 00:42:45.159518 8124 leader_election.cc:334] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589): Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
W20260809 00:42:45.159582 8385 leader_election.cc:334] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589): Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:45.161931 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.158230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:45939 (local address 127.0.0.1:56814)
0809 00:42:45.158369 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.158381 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:45.158441 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:45.159010 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:45.159012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:45.159014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:45.159081 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:45.159084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:45.160172 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:45.160173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:45.161085 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:45.161089 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:45.161696 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:45.161697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:45.161756 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:45.161761 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:45.161815 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 00:42:45.161857 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260809 00:42:45.161963 9160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.158371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.30.104.252:45939)
0809 00:42:45.158577 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:45.158579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:45.158615 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:42:45.158647 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:45.158647 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:45.158660 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:45.158753 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:45.159158 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:45.160101 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:45.161179 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:45.161621 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:45.161677 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:45.161684 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:45.161837 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:42:45.161881 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":93,"threads_started":1}
I20260809 00:42:45.162076 9161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.158593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:37041 (local address 127.0.0.1:47234)
0809 00:42:45.158765 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.158776 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:45.158875 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:45.159587 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:45.159588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:45.159590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:45.159631 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:45.159632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:45.160430 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:45.160431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:45.161266 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:45.161268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:45.161840 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:45.161841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:45.161897 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:45.161901 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:45.161960 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:42:45.162002 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":53,"threads_started":1}
I20260809 00:42:45.162122 9163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.158648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.30.104.253:37041)
0809 00:42:45.158837 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:45.158839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:45.158847 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:45.159226 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:45.159227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:45.159238 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:45.159498 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:45.159717 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:45.160340 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:45.161350 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:45.161749 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:45.161823 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:45.161830 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:45.161976 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 00:42:45.162018 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":49,"threads_started":1}
I20260809 00:42:45.162328 8158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "406dea23b8464ffd88e6881c301c3094" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "82e929c186674536a7841a1dc3b681b5" is_pre_election: true
I20260809 00:42:45.162422 8432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82e929c186674536a7841a1dc3b681b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "406dea23b8464ffd88e6881c301c3094" is_pre_election: true
I20260809 00:42:45.162462 8158 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 406dea23b8464ffd88e6881c301c3094 in term 1.
I20260809 00:42:45.162509 8432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82e929c186674536a7841a1dc3b681b5 in term 1.
I20260809 00:42:45.162653 8386 leader_election.cc:302] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 406dea23b8464ffd88e6881c301c3094, 82e929c186674536a7841a1dc3b681b5; no voters: 1f8a3704ece84f08a8e873f37aba8dcb
I20260809 00:42:45.162781 9157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 00:42:45.162829 9157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f8a3704ece84f08a8e873f37aba8dcb)
I20260809 00:42:45.162855 9157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:42:45.162920 8124 leader_election.cc:302] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 406dea23b8464ffd88e6881c301c3094, 82e929c186674536a7841a1dc3b681b5; no voters: 1f8a3704ece84f08a8e873f37aba8dcb
I20260809 00:42:45.163067 9156 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 00:42:45.163112 9156 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1f8a3704ece84f08a8e873f37aba8dcb)
I20260809 00:42:45.163144 9156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:42:45.163779 9157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } }
I20260809 00:42:45.163945 9157 leader_election.cc:288] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [CANDIDATE]: Term 2 election: Requested vote from peers 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589), 82e929c186674536a7841a1dc3b681b5 (127.30.104.253:37041)
I20260809 00:42:45.163980 9156 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } }
I20260809 00:42:45.164127 9156 leader_election.cc:288] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 2 election: Requested vote from peers 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589), 406dea23b8464ffd88e6881c301c3094 (127.30.104.252:45939)
I20260809 00:42:45.164212 9161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.164031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42968)
0809 00:42:45.164129 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.164182 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 00:42:45.164400 8432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82e929c186674536a7841a1dc3b681b5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "406dea23b8464ffd88e6881c301c3094"
I20260809 00:42:45.164330 8158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "406dea23b8464ffd88e6881c301c3094" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "82e929c186674536a7841a1dc3b681b5"
I20260809 00:42:45.164489 8432 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 82e929c186674536a7841a1dc3b681b5 in current term 2: Already voted for candidate 406dea23b8464ffd88e6881c301c3094 in this term.
I20260809 00:42:45.164506 8158 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 406dea23b8464ffd88e6881c301c3094 in current term 2: Already voted for candidate 82e929c186674536a7841a1dc3b681b5 in this term.
I20260809 00:42:45.164526 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.164409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42996)
0809 00:42:45.164470 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.164498 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 00:42:45.164638 8385 leader_election.cc:334] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589): Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:45.164713 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.164200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:42980)
0809 00:42:45.164653 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.164687 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437}
I20260809 00:42:45.164700 8386 leader_election.cc:302] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 406dea23b8464ffd88e6881c301c3094; no voters: 1f8a3704ece84f08a8e873f37aba8dcb, 82e929c186674536a7841a1dc3b681b5
I20260809 00:42:45.164803 9157 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 00:42:45.165062 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.164960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:43012)
0809 00:42:45.165008 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.165038 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:42:45.165164 8124 leader_election.cc:334] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589): Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:45.165215 8124 leader_election.cc:302] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82e929c186674536a7841a1dc3b681b5; no voters: 1f8a3704ece84f08a8e873f37aba8dcb, 406dea23b8464ffd88e6881c301c3094
I20260809 00:42:45.165306 9156 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 00:42:45.656605 9167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.656314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:57405)
0809 00:42:45.656460 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.656534 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260809 00:42:45.659607 9169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.659344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:49807)
0809 00:42:45.659522 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.659568 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":73,"threads_started":1}
I20260809 00:42:45.660544 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:45.660362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:50593)
0809 00:42:45.660471 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:45.660509 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260809 00:42:46.657402 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.657127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:39499)
0809 00:42:46.657275 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.657326 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":73,"threads_started":1}
I20260809 00:42:46.660313 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.660098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:48959)
0809 00:42:46.660220 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.660261 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260809 00:42:46.661198 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.660997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:40227)
0809 00:42:46.661123 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.661161 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":53,"threads_started":1}
I20260809 00:42:46.702034 9177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:46.702184 9177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } }
I20260809 00:42:46.702395 9177 leader_election.cc:288] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589), 406dea23b8464ffd88e6881c301c3094 (127.30.104.252:45939)
I20260809 00:42:46.702731 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.702524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:59008)
0809 00:42:46.702640 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.702693 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":52,"threads_started":1}
I20260809 00:42:46.702821 8432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82e929c186674536a7841a1dc3b681b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "406dea23b8464ffd88e6881c301c3094" is_pre_election: true
I20260809 00:42:46.702929 8432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82e929c186674536a7841a1dc3b681b5 in term 2.
I20260809 00:42:46.703225 8124 leader_election.cc:302] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 406dea23b8464ffd88e6881c301c3094, 82e929c186674536a7841a1dc3b681b5; no voters:
I20260809 00:42:46.703322 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.703180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:59024)
0809 00:42:46.703235 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.703267 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 00:42:46.703469 8124 leader_election.cc:334] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589): Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:46.703410 9177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 00:42:46.703542 9177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:46.703572 9177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:42:46.704299 9177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } }
I20260809 00:42:46.704442 9177 leader_election.cc:288] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 3 election: Requested vote from peers 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589), 406dea23b8464ffd88e6881c301c3094 (127.30.104.252:45939)
I20260809 00:42:46.704633 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.704528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:59040)
0809 00:42:46.704574 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.704609 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 00:42:46.704738 8432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82e929c186674536a7841a1dc3b681b5" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "406dea23b8464ffd88e6881c301c3094"
I20260809 00:42:46.704833 8432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:42:46.704986 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.704881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:59046)
0809 00:42:46.704913 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.704938 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 00:42:46.705098 8124 leader_election.cc:334] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589): Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
I20260809 00:42:46.705653 8432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82e929c186674536a7841a1dc3b681b5 in term 3.
I20260809 00:42:46.705847 8124 leader_election.cc:302] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 406dea23b8464ffd88e6881c301c3094, 82e929c186674536a7841a1dc3b681b5; no voters: 1f8a3704ece84f08a8e873f37aba8dcb
I20260809 00:42:46.705958 9177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 00:42:46.706091 9177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [term 3 LEADER]: Becoming Leader. State: Replica: 82e929c186674536a7841a1dc3b681b5, State: Running, Role: LEADER
I20260809 00:42:46.706185 9177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } }
I20260809 00:42:46.706635 9180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e929c186674536a7841a1dc3b681b5. Latest consensus state: current_term: 3 leader_uuid: "82e929c186674536a7841a1dc3b681b5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:46.706744 9180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:46.706959 9185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:46.707396 9185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:46.707547 9185 catalog_manager.cc:1259] Loaded cluster ID: 36fc3005ae8f4261a5790d5d99715fd0
I20260809 00:42:46.707587 9185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:46.707782 9185 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:46.707872 9185 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Loaded TSK: 0
I20260809 00:42:46.708056 9185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:42:46.782030 8145 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47228:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:46.783749 8145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:46.785156 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.784970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:59074)
0809 00:42:46.785082 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.785120 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 00:42:46.785315 8432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [term 3 FOLLOWER]: Refusing update from remote peer 82e929c186674536a7841a1dc3b681b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260809 00:42:46.785552 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.785445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.0.0.1:59082)
0809 00:42:46.785478 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.785534 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 00:42:46.785627 9180 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 00:42:46.785804 8124 consensus_peers.cc:487] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 -> Peer 1f8a3704ece84f08a8e873f37aba8dcb (127.30.104.254:40589): Couldn't send request to peer 1f8a3704ece84f08a8e873f37aba8dcb. Status: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 00:42:46.786353 9188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82e929c186674536a7841a1dc3b681b5. Latest consensus state: current_term: 3 leader_uuid: "82e929c186674536a7841a1dc3b681b5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:46.786471 9188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:46.786944 9187 mvcc.cc:205] Tried to move back new op lower bound from 7316423339148816384 to 7316423338829332480. Current Snapshot: MvccSnapshot[applied={T|T < 7316423339148816384}]
I20260809 00:42:46.787056 9177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "82e929c186674536a7841a1dc3b681b5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:46.787196 9177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:46.787389 9188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "82e929c186674536a7841a1dc3b681b5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "1f8a3704ece84f08a8e873f37aba8dcb" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 40589 } } peers { permanent_uuid: "82e929c186674536a7841a1dc3b681b5" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 37041 } attrs { promote: false } } peers { permanent_uuid: "406dea23b8464ffd88e6881c301c3094" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 45939 } attrs { promote: false } } }
I20260809 00:42:46.787526 9188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 406dea23b8464ffd88e6881c301c3094 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:42:46.793318 9178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.789280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44235 (local address 127.0.0.1:39412)
0809 00:42:46.789338 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.789350 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:46.789411 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:46.789857 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:46.789858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:46.789859 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:46.789936 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:46.789938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.790995 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.790996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:46.792289 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:46.792292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.793084 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.793085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:46.793155 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:46.793161 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:42:46.793193 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 00:42:46.793236 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":95}
I20260809 00:42:46.793426 9203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.789453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.30.104.195:33781)
0809 00:42:46.789580 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:46.789583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:46.789639 (+ 56us) server_negotiation.cc:407] Connection header received
0809 00:42:46.789713 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:46.789714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:46.789726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:46.789797 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:46.790045 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.790877 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.792631 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.793042 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.793131 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:46.793142 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:46.793291 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 00:42:46.793341 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260809 00:42:46.793428 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.789375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:33781 (local address 127.0.0.1:49774)
0809 00:42:46.789585 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.789595 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:46.789649 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:46.789857 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:46.789858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:46.789861 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:46.789933 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:46.789935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.790995 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.790996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:46.792544 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:46.792546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.793137 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.793138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:46.793208 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:46.793211 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:46.793272 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:42:46.793317 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":43,"thread_start_us":97,"threads_started":1}
I20260809 00:42:46.793840 8965 tablet_service.cc:1467] Processing CreateTablet for tablet 29ebd2663d0349ffab582ac44bf4c862 (DEFAULT_TABLE table=test-table [id=b9cb4287ffea4a4ab92335af964fb9cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:46.794006 8965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ebd2663d0349ffab582ac44bf4c862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:46.794118 9204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.789605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33821 (local address 127.0.0.1:59434)
0809 00:42:46.791022 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.791035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:46.791119 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:46.791255 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:46.791255 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:46.791257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:46.791304 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:46.791306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.792051 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.792052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:46.793083 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:46.793085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.793881 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.793882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:46.793943 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:46.793947 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:46.794006 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:42:46.794046 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1394,"thread_start_us":71,"threads_started":1}
I20260809 00:42:46.794163 9200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.789393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.30.104.194:44235)
0809 00:42:46.789745 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:46.789748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:46.789756 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:46.789782 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:46.789783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:46.789794 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:46.789856 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:46.790025 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.790877 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.792385 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.792979 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.794004 (+ 1025us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:46.794016 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:46.794035 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 00:42:46.794084 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":77,"threads_started":1}
I20260809 00:42:46.794369 9206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.789785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59434 (local address 127.30.104.193:33821)
0809 00:42:46.789962 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:46.789965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:46.791110 (+ 1145us) server_negotiation.cc:407] Connection header received
0809 00:42:46.791176 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:46.791177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:46.791188 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:46.791254 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:46.791356 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.791962 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.793171 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:46.793778 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:46.793840 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:46.793847 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:46.794225 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 00:42:46.794284 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":62,"threads_started":1}
I20260809 00:42:46.794534 8774 tablet_service.cc:1467] Processing CreateTablet for tablet 29ebd2663d0349ffab582ac44bf4c862 (DEFAULT_TABLE table=test-table [id=b9cb4287ffea4a4ab92335af964fb9cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:46.794660 8774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ebd2663d0349ffab582ac44bf4c862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:46.794664 8574 tablet_service.cc:1467] Processing CreateTablet for tablet 29ebd2663d0349ffab582ac44bf4c862 (DEFAULT_TABLE table=test-table [id=b9cb4287ffea4a4ab92335af964fb9cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:46.794796 8574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29ebd2663d0349ffab582ac44bf4c862. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:46.795849 9208 tablet_bootstrap.cc:492] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: Bootstrap starting.
I20260809 00:42:46.796264 9208 tablet_bootstrap.cc:654] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:46.796453 9209 tablet_bootstrap.cc:492] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92: Bootstrap starting.
I20260809 00:42:46.796620 9207 tablet_bootstrap.cc:492] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: Bootstrap starting.
I20260809 00:42:46.796885 9209 tablet_bootstrap.cc:654] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:46.797046 9207 tablet_bootstrap.cc:654] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:46.797111 9208 tablet_bootstrap.cc:492] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: No bootstrap required, opened a new log
I20260809 00:42:46.797183 9208 ts_tablet_manager.cc:1397] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:46.797360 9208 raft_consensus.cc:348] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:46.797459 9208 raft_consensus.cc:374] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:46.797483 9208 raft_consensus.cc:729] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6da8911a57e4f19958a580c25f21555, State: Initialized, Role: FOLLOWER
I20260809 00:42:46.797529 9209 tablet_bootstrap.cc:492] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92: No bootstrap required, opened a new log
I20260809 00:42:46.797557 9208 consensus_queue.cc:260] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:46.797632 9209 ts_tablet_manager.cc:1397] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:46.797776 9208 ts_tablet_manager.cc:1428] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:46.797873 9207 tablet_bootstrap.cc:492] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: No bootstrap required, opened a new log
I20260809 00:42:46.797794 9209 raft_consensus.cc:348] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:46.798007 9209 raft_consensus.cc:374] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:46.798053 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.797940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:46227)
0809 00:42:46.797984 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.798023 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 00:42:46.798067 9209 raft_consensus.cc:729] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb53d311bae2400096cd1463ff086f92, State: Initialized, Role: FOLLOWER
I20260809 00:42:46.798115 9209 consensus_queue.cc:260] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:46.798310 9209 ts_tablet_manager.cc:1428] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:46.798437 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.798327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:44713)
0809 00:42:46.798379 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.798406 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 00:42:46.798579 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.798471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:55889)
0809 00:42:46.798523 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.798556 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 00:42:46.798911 8878 heartbeater.cc:503] Master 127.30.104.253:37041 was elected leader, sending a full tablet report...
I20260809 00:42:46.798936 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.798821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:42781)
0809 00:42:46.798876 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.798907 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:42:46.797930 9207 ts_tablet_manager.cc:1397] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:46.799623 9207 raft_consensus.cc:348] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:46.799706 9207 raft_consensus.cc:374] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:46.799726 9207 raft_consensus.cc:729] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24133fd916f249f291f167baf4e2974a, State: Initialized, Role: FOLLOWER
I20260809 00:42:46.799801 9207 consensus_queue.cc:260] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:46.799988 9207 ts_tablet_manager.cc:1428] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:46.800330 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.800198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:55185)
0809 00:42:46.800242 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.800297 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 00:42:46.800560 9058 heartbeater.cc:503] Master 127.30.104.253:37041 was elected leader, sending a full tablet report...
I20260809 00:42:46.800653 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:46.800557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:43485)
0809 00:42:46.800597 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:46.800626 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 00:42:47.100800 9218 raft_consensus.cc:482] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:47.100953 9218 raft_consensus.cc:504] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:47.101342 9218 leader_election.cc:288] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235), fb53d311bae2400096cd1463ff086f92 (127.30.104.193:33821)
I20260809 00:42:47.105700 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.101712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44235 (local address 127.30.104.195:48925)
0809 00:42:47.101788 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.101803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:47.101904 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:47.102102 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:47.102103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:47.102105 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:47.102180 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:47.102182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.103155 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.103156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:47.104550 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:47.104554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.105322 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.105323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:47.105430 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:47.105436 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:42:47.105508 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 00:42:47.105566 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":121}
I20260809 00:42:47.105731 9200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.101837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:48925 (local address 127.30.104.194:44235)
0809 00:42:47.101895 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:47.101897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:47.101912 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:42:47.101986 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:47.101987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:47.101998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:47.102064 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:47.102258 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.103026 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.104669 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.105234 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.105314 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:47.105327 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:47.105590 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 00:42:47.105646 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:42:47.105731 9206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.101935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:58503 (local address 127.30.104.193:33821)
0809 00:42:47.102005 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:47.102007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:47.102160 (+ 153us) server_negotiation.cc:407] Connection header received
0809 00:42:47.102193 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:47.102194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:47.102203 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:47.102274 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:47.102395 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.103088 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.104717 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.105304 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.105371 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:47.105382 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:47.105591 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 00:42:47.105645 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:42:47.105705 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.101836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33821 (local address 127.30.104.195:58503)
0809 00:42:47.102055 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.102065 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:47.102118 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:47.102297 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:47.102297 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:47.102299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:47.102336 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:47.102338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.103179 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.103179 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:47.104550 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:47.104554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.105393 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.105394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:47.105460 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:47.105464 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:47.105522 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 00:42:47.105566 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"mutex_wait_us":11,"thread_start_us":127,"threads_started":1}
I20260809 00:42:47.106200 8809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ebd2663d0349ffab582ac44bf4c862" candidate_uuid: "24133fd916f249f291f167baf4e2974a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6da8911a57e4f19958a580c25f21555" is_pre_election: true
I20260809 00:42:47.106299 8809 raft_consensus.cc:2454] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24133fd916f249f291f167baf4e2974a in term 0.
I20260809 00:42:47.106226 8605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ebd2663d0349ffab582ac44bf4c862" candidate_uuid: "24133fd916f249f291f167baf4e2974a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb53d311bae2400096cd1463ff086f92" is_pre_election: true
I20260809 00:42:47.106415 8605 raft_consensus.cc:2454] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24133fd916f249f291f167baf4e2974a in term 0.
I20260809 00:42:47.106508 8907 leader_election.cc:302] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24133fd916f249f291f167baf4e2974a, b6da8911a57e4f19958a580c25f21555; no voters:
I20260809 00:42:47.106662 9218 raft_consensus.cc:2781] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:47.106709 9218 raft_consensus.cc:482] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:47.106732 9218 raft_consensus.cc:3037] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:47.107492 9218 raft_consensus.cc:504] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:47.107645 9218 leader_election.cc:288] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [CANDIDATE]: Term 1 election: Requested vote from peers b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235), fb53d311bae2400096cd1463ff086f92 (127.30.104.193:33821)
I20260809 00:42:47.107841 8605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ebd2663d0349ffab582ac44bf4c862" candidate_uuid: "24133fd916f249f291f167baf4e2974a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb53d311bae2400096cd1463ff086f92"
I20260809 00:42:47.107933 8605 raft_consensus.cc:3037] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:47.108662 8605 raft_consensus.cc:2454] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24133fd916f249f291f167baf4e2974a in term 1.
I20260809 00:42:47.108796 8809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29ebd2663d0349ffab582ac44bf4c862" candidate_uuid: "24133fd916f249f291f167baf4e2974a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6da8911a57e4f19958a580c25f21555"
I20260809 00:42:47.108908 8809 raft_consensus.cc:3037] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:47.108922 8907 leader_election.cc:302] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24133fd916f249f291f167baf4e2974a, fb53d311bae2400096cd1463ff086f92; no voters:
I20260809 00:42:47.109033 9218 raft_consensus.cc:2781] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:47.109272 9218 raft_consensus.cc:686] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 1 LEADER]: Becoming Leader. State: Replica: 24133fd916f249f291f167baf4e2974a, State: Running, Role: LEADER
I20260809 00:42:47.109362 9218 consensus_queue.cc:237] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } }
I20260809 00:42:47.109809 8809 raft_consensus.cc:2454] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24133fd916f249f291f167baf4e2974a in term 1.
I20260809 00:42:47.109835 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.109668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:35937)
0809 00:42:47.109771 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.109808 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 00:42:47.110819 8146 catalog_manager.cc:5495] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a reported cstate change: term changed from 0 to 1, leader changed from <none> to 24133fd916f249f291f167baf4e2974a (127.30.104.195). New cstate: current_term: 1 leader_uuid: "24133fd916f249f291f167baf4e2974a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24133fd916f249f291f167baf4e2974a" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33781 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:47.150704 9203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.148152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.30.104.195:33781)
0809 00:42:47.148218 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:47.148220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:47.148230 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:47.148282 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:47.148283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:47.148302 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:47.148412 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:47.148575 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.149078 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.149873 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:47.150078 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:47.150157 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:47.150340 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:47.150453 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:47.150541 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:47.150583 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 00:42:47.150607 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:42:47.151994 8809 raft_consensus.cc:1261] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 1 FOLLOWER]: Refusing update from remote peer 24133fd916f249f291f167baf4e2974a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:47.152033 8605 raft_consensus.cc:1261] T 29ebd2663d0349ffab582ac44bf4c862 P fb53d311bae2400096cd1463ff086f92 [term 1 FOLLOWER]: Refusing update from remote peer 24133fd916f249f291f167baf4e2974a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:47.152293 9220 consensus_queue.cc:1035] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6da8911a57e4f19958a580c25f21555" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:47.152477 9218 consensus_queue.cc:1035] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb53d311bae2400096cd1463ff086f92" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 33821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:47.153541 8682 heartbeater.cc:503] Master 127.30.104.253:37041 was elected leader, sending a full tablet report...
I20260809 00:42:47.153885 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.153666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:48891)
0809 00:42:47.153802 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.153841 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 00:42:47.154836 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.154691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:57485)
0809 00:42:47.154772 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.154802 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 00:42:47.156080 9230 mvcc.cc:205] Tried to move back new op lower bound from 7316423340651888640 to 7316423340482805760. Current Snapshot: MvccSnapshot[applied={T|T < 7316423340651888640 or (T in {7316423340651888640})}]
I20260809 00:42:47.156277 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.156150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.194:52891)
0809 00:42:47.156203 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.156235 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:42:47.156761 9176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.156641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.193:55285)
0809 00:42:47.156707 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.156734 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 00:42:47.157310 8146 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:47.157407 8146 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47228:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:47.157382 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.154906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:59509)
0809 00:42:47.157317 (+ 2411us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.157357 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2394}
I20260809 00:42:47.157711 9219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.157630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:40589 (local address 127.30.104.195:36157)
0809 00:42:47.157661 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.157686 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:40589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 00:42:47.158856 8146 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 82e929c186674536a7841a1dc3b681b5: Sending DeleteTablet for 3 replicas of tablet 29ebd2663d0349ffab582ac44bf4c862
I20260809 00:42:47.159179 8774 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ebd2663d0349ffab582ac44bf4c862 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:47 UTC) from {username='oryx', principal='oryx/127.30.104.253@KRBTEST.COM'} at 127.0.0.1:39412
I20260809 00:42:47.159247 8965 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ebd2663d0349ffab582ac44bf4c862 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:47 UTC) from {username='oryx', principal='oryx/127.30.104.253@KRBTEST.COM'} at 127.0.0.1:49774
I20260809 00:42:47.159422 9237 tablet_replica.cc:330] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: stopping tablet replica
I20260809 00:42:47.159422 8574 tablet_service.cc:1514] Processing DeleteTablet for tablet 29ebd2663d0349ffab582ac44bf4c862 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:47 UTC) from {username='oryx', principal='oryx/127.30.104.253@KRBTEST.COM'} at 127.0.0.1:59434
I20260809 00:42:47.159512 9237 raft_consensus.cc:2229] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:47.159557 9237 raft_consensus.cc:2258] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:47.159768 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 8355
I20260809 00:42:47.159790 9238 tablet_replica.cc:330] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: stopping tablet replica
I20260809 00:42:47.159848 9238 raft_consensus.cc:2229] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:47.162101 9238 raft_consensus.cc:2258] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:47.173791 9237 ts_tablet_manager.cc:1905] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:47.175400 9237 ts_tablet_manager.cc:1918] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:47.175604 9237 log.cc:1196] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/29ebd2663d0349ffab582ac44bf4c862
I20260809 00:42:47.175757 9238 ts_tablet_manager.cc:1905] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:47.175902 9237 ts_tablet_manager.cc:1939] T 29ebd2663d0349ffab582ac44bf4c862 P b6da8911a57e4f19958a580c25f21555: Deleting consensus metadata
I20260809 00:42:47.176426 8125 catalog_manager.cc:4841] TS b6da8911a57e4f19958a580c25f21555 (127.30.104.194:44235): tablet 29ebd2663d0349ffab582ac44bf4c862 (table test-table [id=b9cb4287ffea4a4ab92335af964fb9cf]) successfully deleted
I20260809 00:42:47.177364 9238 ts_tablet_manager.cc:1918] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:47.177433 9238 log.cc:1196] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/29ebd2663d0349ffab582ac44bf4c862
I20260809 00:42:47.177706 9238 ts_tablet_manager.cc:1939] T 29ebd2663d0349ffab582ac44bf4c862 P 24133fd916f249f291f167baf4e2974a: Deleting consensus metadata
I20260809 00:42:47.178097 8125 catalog_manager.cc:4841] TS 24133fd916f249f291f167baf4e2974a (127.30.104.195:33781): tablet 29ebd2663d0349ffab582ac44bf4c862 (table test-table [id=b9cb4287ffea4a4ab92335af964fb9cf]) successfully deleted
I20260809 00:42:47.184432 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 8689
I20260809 00:42:47.184533 9204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:47.184331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:33821 (local address 127.0.0.1:59436)
0809 00:42:47.184410 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:47.184449 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:33821: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 00:42:47.192478 8125 catalog_manager.cc:4584] TS fb53d311bae2400096cd1463ff086f92 (127.30.104.193:33821): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 29ebd2663d0349ffab582ac44bf4c862: Network error: Client connection negotiation failed: client connection to 127.30.104.193:33821: connect: Connection refused (error 111)
I20260809 00:42:47.198525 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 8884
I20260809 00:42:47.210939 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 7962
I20260809 00:42:47.221984 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 8221
2026-08-09T00:42:47Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7285 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:47 dist-test-slave-c12x krb5kdc[9243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:47 dist-test-slave-c12x krb5kdc[9243](info): set up 2 sockets
Aug 09 00:42:47 dist-test-slave-c12x krb5kdc[9243](info): commencing operation
krb5kdc: starting...
W20260809 00:42:48.475860 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:48 dist-test-slave-c12x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.254.keytab.
Entry for principal oryx/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.254.keytab.
I20260809 00:42:48.607380 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:41501
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.254.keytab
--principal=oryx/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:45451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:41501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:48.693747 9264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:48.693990 9264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:48.694028 9264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:48.695184 9264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:48.695252 9264 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 00:42:48.695277 9264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:48.695312 9264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:48.695343 9264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:48.695369 9264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:48.696973 9264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:45451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:41501
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.254.keytab
--principal=oryx/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:41501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:48.697265 9264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:48.697503 9264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:48.702535 9270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:48.702538 9274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:48.702703 9264 server_base.cc:1034] running on GCE node
W20260809 00:42:48.702659 9271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:48.703197 9264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:48.703504 9264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:48.704661 9264 hybrid_clock.cc:648] HybridClock initialized: now 1786236168704630 us; error 51 us; skew 500 ppm
Aug 09 00:42:48 dist-test-slave-c12x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236168, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:48.707193 9264 init.cc:377] Logged in from keytab as oryx/127.30.104.254@KRBTEST.COM (short username oryx)
I20260809 00:42:48.707721 9264 webserver.cc:460] Webserver started at http://127.30.104.254:41599/ using document root <none> and password file <none>
I20260809 00:42:48.707979 9264 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:48.708056 9264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:48.708215 9264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:48.709324 9264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "5e8ef91760bc459b93e605da798a67a5"
format_stamp: "Formatted at 2026-08-09 00:42:48 on dist-test-slave-c12x"
server_key: "aabd7245217a43581e003099853ef145"
server_key_iv: "df6704aa1ac9e7ca841c8c9d1ecbad0b"
server_key_version: "encryptionkey@0"
I20260809 00:42:48.709710 9264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "5e8ef91760bc459b93e605da798a67a5"
format_stamp: "Formatted at 2026-08-09 00:42:48 on dist-test-slave-c12x"
server_key: "aabd7245217a43581e003099853ef145"
server_key_iv: "df6704aa1ac9e7ca841c8c9d1ecbad0b"
server_key_version: "encryptionkey@0"
I20260809 00:42:48.711109 9264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:42:48.711963 9290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:48.712180 9264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:48.712302 9264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "5e8ef91760bc459b93e605da798a67a5"
format_stamp: "Formatted at 2026-08-09 00:42:48 on dist-test-slave-c12x"
server_key: "aabd7245217a43581e003099853ef145"
server_key_iv: "df6704aa1ac9e7ca841c8c9d1ecbad0b"
server_key_version: "encryptionkey@0"
I20260809 00:42:48.712393 9264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:48.723887 9264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:48.725955 9264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:48.726130 9264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:48.731384 9264 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:41501
I20260809 00:42:48.731426 9380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:41501 every 8 connection(s)
I20260809 00:42:48.731793 9264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:48.732491 9382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:48.733358 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 9264
I20260809 00:42:48.733440 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:48.733696 31139 external_mini_cluster.cc:1428] Setting key 8097586f0b506972342a1ab3af14db6f
I20260809 00:42:48.735294 9382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: Bootstrap starting.
I20260809 00:42:48.735983 9382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:48.736302 9382 log.cc:824] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:48.737011 9382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: No bootstrap required, opened a new log
I20260809 00:42:48.738860 9382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8ef91760bc459b93e605da798a67a5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41501 } }
I20260809 00:42:48.739030 9382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:48.739084 9382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e8ef91760bc459b93e605da798a67a5, State: Initialized, Role: FOLLOWER
I20260809 00:42:48.739241 9382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8ef91760bc459b93e605da798a67a5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41501 } }
I20260809 00:42:48.739339 9382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:48.739385 9382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:48.739460 9382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:48.740052 9382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8ef91760bc459b93e605da798a67a5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41501 } }
I20260809 00:42:48.740176 9382 leader_election.cc:302] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e8ef91760bc459b93e605da798a67a5; no voters:
I20260809 00:42:48.740355 9382 leader_election.cc:288] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:48.740412 9388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 1 FOLLOWER]: Leader election won for term 1
Aug 09 00:42:48 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:48.740621 9382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:48.740643 9388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [term 1 LEADER]: Becoming Leader. State: Replica: 5e8ef91760bc459b93e605da798a67a5, State: Running, Role: LEADER
I20260809 00:42:48.740733 9388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8ef91760bc459b93e605da798a67a5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41501 } }
I20260809 00:42:48.741110 9389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e8ef91760bc459b93e605da798a67a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8ef91760bc459b93e605da798a67a5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41501 } } }
I20260809 00:42:48.741102 9388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e8ef91760bc459b93e605da798a67a5. Latest consensus state: current_term: 1 leader_uuid: "5e8ef91760bc459b93e605da798a67a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e8ef91760bc459b93e605da798a67a5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41501 } } }
I20260809 00:42:48.741209 9389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:48.741216 9388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5 [sys.catalog]: This master's current role is: LEADER
W20260809 00:42:48.742817 9406 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:42:48.742877 9406 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:42:48.742931 9398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:48.743098 9398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:48.744777 9398 catalog_manager.cc:1347] Generated new cluster ID: d39d1420d5a54be4bf4073897a221a34
I20260809 00:42:48.744837 9398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:48.744975 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:48.734250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46096 (local address 127.30.104.254:41501)
0809 00:42:48.734444 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:48.734454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:48.734490 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:42:48.735151 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:48.735159 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:48.735209 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:48.735360 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:48.736042 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.737027 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.737785 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.738209 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.738332 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:48.741237 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:48.741241 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:48.741242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:48.741256 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:48.743264 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:48.743690 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:48.743692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:48.743693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:48.743775 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:48.744019 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:48.744020 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:48.744021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:48.744353 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:48.744502 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:48.744624 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 00:42:48.744765 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":90,"threads_started":1}
I20260809 00:42:48.751475 9398 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:48.752184 9398 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:48.757259 9398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: Generated new TSK 0
I20260809 00:42:48.757439 9398 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab.
Entry for principal oryx/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab.
I20260809 00:42:48.795117 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41501
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab
--principal=oryx/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:45451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:48.887282 9412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:48.887586 9412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:48.887617 9412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:48.888863 9412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:48.888913 9412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:48.888936 9412 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:48.888954 9412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:48.890574 9412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:45451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab
--principal=oryx/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:48.890882 9412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:48.891144 9412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:48.897383 9420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:48.897385 9422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:48.897387 9418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:48.897734 9412 server_base.cc:1034] running on GCE node
I20260809 00:42:48.897964 9412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:48.898223 9412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:48.899501 9412 hybrid_clock.cc:648] HybridClock initialized: now 1786236168899384 us; error 39 us; skew 500 ppm
Aug 09 00:42:48 dist-test-slave-c12x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236168, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:48.902277 9412 init.cc:377] Logged in from keytab as oryx/127.30.104.193@KRBTEST.COM (short username oryx)
I20260809 00:42:48.902838 9412 webserver.cc:460] Webserver started at http://127.30.104.193:46159/ using document root <none> and password file <none>
I20260809 00:42:48.903103 9412 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:48.903177 9412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:48.903330 9412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:48.904506 9412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "0d77fabd64134dcab7f8e2566ef79351"
format_stamp: "Formatted at 2026-08-09 00:42:48 on dist-test-slave-c12x"
server_key: "6ecaab77a7499f58b61ced66ba364fb7"
server_key_iv: "5e93db8511a605e91074487a9e1b2d50"
server_key_version: "encryptionkey@0"
I20260809 00:42:48.904892 9412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "0d77fabd64134dcab7f8e2566ef79351"
format_stamp: "Formatted at 2026-08-09 00:42:48 on dist-test-slave-c12x"
server_key: "6ecaab77a7499f58b61ced66ba364fb7"
server_key_iv: "5e93db8511a605e91074487a9e1b2d50"
server_key_version: "encryptionkey@0"
I20260809 00:42:48.906390 9412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:48.907264 9432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:48.907560 9412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:48.907691 9412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "0d77fabd64134dcab7f8e2566ef79351"
format_stamp: "Formatted at 2026-08-09 00:42:48 on dist-test-slave-c12x"
server_key: "6ecaab77a7499f58b61ced66ba364fb7"
server_key_iv: "5e93db8511a605e91074487a9e1b2d50"
server_key_version: "encryptionkey@0"
I20260809 00:42:48.907788 9412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:48.921092 9412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:48.923592 9412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:48.923789 9412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:48.924506 9412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:48.924562 9412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:48.924611 9412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:48.924635 9412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:42:48 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236168, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:48.932277 9412 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:43601
I20260809 00:42:48.932354 9585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:43601 every 8 connection(s)
I20260809 00:42:48.932705 9412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:48.935336 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:48.925055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:47495 (local address 127.30.104.254:41501)
0809 00:42:48.925137 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:48.925141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:48.925384 (+ 243us) server_negotiation.cc:407] Connection header received
0809 00:42:48.925975 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:48.925977 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:48.925992 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:48.926078 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:48.927126 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.928014 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.929145 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.929371 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.929450 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:48.932336 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:48.932338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:48.932338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:48.932348 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:48.933984 (+ 1636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:48.934429 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:48.934430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:48.934430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:48.934512 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:48.934764 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:48.934765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:48.934765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:48.934895 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:48.934967 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:48.935196 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 00:42:48.935270 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:42:48.935628 9446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:48.925050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41501 (local address 127.30.104.193:47495)
0809 00:42:48.925234 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:48.925277 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:48.925833 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:48.926182 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:48.926186 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:48.926663 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:48.927010 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:48.927017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.928128 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.928129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:48.929016 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:48.929019 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.929484 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.929486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:48.929619 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:48.930300 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:48.930310 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:48.932241 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:48.934084 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:48.934091 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:48.934099 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:48.934326 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:48.934556 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:48.934557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:48.934557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:48.934651 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:48.934967 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:48.934970 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:48.935090 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 00:42:48.935173 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:42:48.935299 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260809 00:42:48.936209 9588 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41501
I20260809 00:42:48.936301 9588 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:48.936504 9588 heartbeater.cc:511] Master 127.30.104.254:41501 requested a full tablet report, sending...
I20260809 00:42:48.937132 9315 ts_manager.cc:194] Registered new tserver with Master: 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601)
I20260809 00:42:48.937876 9315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:47495
I20260809 00:42:48.942312 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 9412
I20260809 00:42:48.942443 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:48.942672 31139 external_mini_cluster.cc:1428] Setting key 44e0815d8d63b5729c36c74c901c659d
I20260809 00:42:48.944083 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:48.937381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:51369 (local address 127.30.104.254:41501)
0809 00:42:48.937418 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:48.937420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:48.938204 (+ 784us) server_negotiation.cc:407] Connection header received
0809 00:42:48.938268 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:48.938269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:48.938286 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:48.938352 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:48.938747 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.939233 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.939991 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.940187 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.940250 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:48.941269 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:48.941271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:48.941271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:48.941278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:48.942767 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:48.943191 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:48.943192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:48.943193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:48.943252 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:48.943522 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:48.943523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:48.943523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:48.943643 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:48.943718 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:48.943897 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 00:42:48.943977 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:42:48.944094 9596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:48.937333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41501 (local address 127.30.104.193:51369)
0809 00:42:48.938119 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:48.938131 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:48.938192 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:48.938371 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:48.938372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:48.938582 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:48.938645 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:48.938647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.939325 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.939325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:48.939896 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:48.939898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:48.940260 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:48.940261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:48.940308 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:48.940862 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:48.940868 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:48.941178 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:48.942859 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:48.942860 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:48.942861 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:48.943092 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:48.943330 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:48.943331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:48.943331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:48.943405 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:48.943723 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:48.943724 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:48.943773 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 00:42:48.943859 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:42:48.943980 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for oryx/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab.
Entry for principal oryx/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab.
I20260809 00:42:48.971276 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41501
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab
--principal=oryx/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:45451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:49.058594 9600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:49.058801 9600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:49.058826 9600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:49.060055 9600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:49.060099 9600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:49.060122 9600 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:49.060163 9600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:49.061734 9600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:45451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab
--principal=oryx/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:49.061990 9600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:49.062227 9600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:49.067592 9605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:49.067584 9608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:49.067564 9606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:49.067832 9600 server_base.cc:1034] running on GCE node
I20260809 00:42:49.068045 9600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:49.068290 9600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:49.069453 9600 hybrid_clock.cc:648] HybridClock initialized: now 1786236169069420 us; error 52 us; skew 500 ppm
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236169, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:49.072116 9600 init.cc:377] Logged in from keytab as oryx/127.30.104.194@KRBTEST.COM (short username oryx)
I20260809 00:42:49.072618 9600 webserver.cc:460] Webserver started at http://127.30.104.194:41865/ using document root <none> and password file <none>
I20260809 00:42:49.072882 9600 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:49.072929 9600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:49.073097 9600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:49.074177 9600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "5d4e537a618f446cb348d1061782125d"
format_stamp: "Formatted at 2026-08-09 00:42:49 on dist-test-slave-c12x"
server_key: "326a413b94b63565f5b99c78837d884d"
server_key_iv: "7ac9497a0ef9012a44cd6176a8a30151"
server_key_version: "encryptionkey@0"
I20260809 00:42:49.074543 9600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "5d4e537a618f446cb348d1061782125d"
format_stamp: "Formatted at 2026-08-09 00:42:49 on dist-test-slave-c12x"
server_key: "326a413b94b63565f5b99c78837d884d"
server_key_iv: "7ac9497a0ef9012a44cd6176a8a30151"
server_key_version: "encryptionkey@0"
I20260809 00:42:49.076061 9600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:42:49.076921 9620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.077149 9600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:49.077251 9600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "5d4e537a618f446cb348d1061782125d"
format_stamp: "Formatted at 2026-08-09 00:42:49 on dist-test-slave-c12x"
server_key: "326a413b94b63565f5b99c78837d884d"
server_key_iv: "7ac9497a0ef9012a44cd6176a8a30151"
server_key_version: "encryptionkey@0"
I20260809 00:42:49.077330 9600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:49.088857 9600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:49.091024 9600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:49.091184 9600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:49.091869 9600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:49.091912 9600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.091934 9600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:49.091946 9600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236169, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:49.099475 9600 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:41977
I20260809 00:42:49.099601 9776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:41977 every 8 connection(s)
I20260809 00:42:49.099887 9600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:49.102164 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.092296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:50315 (local address 127.30.104.254:41501)
0809 00:42:49.092363 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.092365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.092585 (+ 220us) server_negotiation.cc:407] Connection header received
0809 00:42:49.093254 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.093256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.093271 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.093351 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.093991 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.094844 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.096133 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.096338 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.096428 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.099127 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:49.099129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:49.099129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:49.099136 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:49.100756 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.101261 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.101261 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.101262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.101316 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.101544 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.101544 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.101545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.101653 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:49.101707 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.102020 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.102098 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:42:49.102330 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.092279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41501 (local address 127.30.104.194:50315)
0809 00:42:49.092439 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.092497 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.093095 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.093478 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.093482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.093725 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:49.093910 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.093914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.094990 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.094991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.096033 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.096036 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.096452 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.096453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.096592 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.097160 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:49.097173 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:49.099019 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:49.100853 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.100860 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.100862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.101187 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.101379 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.101380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.101380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.101484 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.101735 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:49.101741 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:49.101881 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.101993 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.102123 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
I20260809 00:42:49.102819 9777 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41501
I20260809 00:42:49.102905 9777 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:49.103132 9777 heartbeater.cc:511] Master 127.30.104.254:41501 requested a full tablet report, sending...
I20260809 00:42:49.103622 9316 ts_manager.cc:194] Registered new tserver with Master: 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977)
I20260809 00:42:49.104055 9316 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:50315
I20260809 00:42:49.108053 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 9600
I20260809 00:42:49.108191 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:49.108433 31139 external_mini_cluster.cc:1428] Setting key 18406b11be9c1f4fdf93b652a957a267
I20260809 00:42:49.110278 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.104365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:37869 (local address 127.30.104.254:41501)
0809 00:42:49.104425 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.104427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.104564 (+ 137us) server_negotiation.cc:407] Connection header received
0809 00:42:49.104639 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.104639 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.104650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.104694 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.105028 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.105497 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.106208 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.106411 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.106490 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.107359 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:49.107360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:49.107360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:49.107367 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:49.109035 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.109441 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.109442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.109442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.109508 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.109750 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.109751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.109751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.109907 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:49.109979 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.110150 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.110217 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:42:49.110388 9788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.104305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41501 (local address 127.30.104.194:37869)
0809 00:42:49.104495 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.104504 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.104589 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.104707 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.104707 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.104899 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:49.104949 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.104951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.105601 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.105602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.106122 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.106123 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.106501 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.106502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.106569 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.106984 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:49.106989 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:49.107265 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:49.109147 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.109148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.109149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.109351 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.109583 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.109585 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.109585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.109659 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.109992 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:49.109993 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:49.110045 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.110119 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.110243 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for oryx/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab.
Entry for principal oryx/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab.
I20260809 00:42:49.136010 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41501
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab
--principal=oryx/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:45451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:49.220031 9791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:49.220264 9791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:49.220305 9791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:49.221490 9791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:49.221560 9791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:49.221597 9791 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 00:42:49.221653 9791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:49.223197 9791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:45451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab
--principal=oryx/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.104.254:41501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:49.223523 9791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:49.223767 9791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:49.229259 9803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:49.229239 9798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:49.229267 9799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:49.229534 9791 server_base.cc:1034] running on GCE node
I20260809 00:42:49.229748 9791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:49.229943 9791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:49.231125 9791 hybrid_clock.cc:648] HybridClock initialized: now 1786236169231102 us; error 34 us; skew 500 ppm
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236169, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:49.233678 9791 init.cc:377] Logged in from keytab as oryx/127.30.104.195@KRBTEST.COM (short username oryx)
I20260809 00:42:49.234135 9791 webserver.cc:460] Webserver started at http://127.30.104.195:45153/ using document root <none> and password file <none>
I20260809 00:42:49.234344 9791 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:49.234385 9791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:49.234483 9791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:49.235554 9791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "2d49785a987d455fbd83095328dc42ae"
format_stamp: "Formatted at 2026-08-09 00:42:49 on dist-test-slave-c12x"
server_key: "0895b5490152f16b8a9a5a0afe5e84ea"
server_key_iv: "79a65cb2677cf20bd0f65271c92fef6c"
server_key_version: "encryptionkey@0"
I20260809 00:42:49.235939 9791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "2d49785a987d455fbd83095328dc42ae"
format_stamp: "Formatted at 2026-08-09 00:42:49 on dist-test-slave-c12x"
server_key: "0895b5490152f16b8a9a5a0afe5e84ea"
server_key_iv: "79a65cb2677cf20bd0f65271c92fef6c"
server_key_version: "encryptionkey@0"
I20260809 00:42:49.237427 9791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:49.238219 9812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.238438 9791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:49.238538 9791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "2d49785a987d455fbd83095328dc42ae"
format_stamp: "Formatted at 2026-08-09 00:42:49 on dist-test-slave-c12x"
server_key: "0895b5490152f16b8a9a5a0afe5e84ea"
server_key_iv: "79a65cb2677cf20bd0f65271c92fef6c"
server_key_version: "encryptionkey@0"
I20260809 00:42:49.238612 9791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:49.254550 9791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:49.256925 9791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:49.257087 9791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:49.257808 9791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:49.257872 9791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.257956 9791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:49.257987 9791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236169, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:49.265412 9791 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:36211
I20260809 00:42:49.265507 9965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:36211 every 8 connection(s)
I20260809 00:42:49.265820 9791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:49.268097 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.258225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:34145 (local address 127.30.104.254:41501)
0809 00:42:49.258280 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.258282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.258537 (+ 255us) server_negotiation.cc:407] Connection header received
0809 00:42:49.259080 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.259082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.259097 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.259185 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.259944 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.260507 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.261553 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.261801 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.261882 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.264861 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:49.264863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:49.264863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:49.264872 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:49.266548 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.267004 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.267006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.267006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.267070 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.267349 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.267350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.267350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.267542 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:49.267621 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.267912 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.268002 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:42:49.268297 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.258236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41501 (local address 127.30.104.195:34145)
0809 00:42:49.258398 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.258439 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.258977 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.259279 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.259283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.259644 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:49.259837 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.259843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.260609 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.260610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.261456 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.261458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.261894 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.261895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.262078 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.262580 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:49.262588 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:49.264751 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:49.266637 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.266644 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.266646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.266923 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.267156 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.267157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.267157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.267272 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.267612 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:49.267619 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:49.267766 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.267880 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.268059 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260809 00:42:49.268807 9966 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41501
I20260809 00:42:49.268900 9966 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:49.269127 9966 heartbeater.cc:511] Master 127.30.104.254:41501 requested a full tablet report, sending...
I20260809 00:42:49.269541 9315 ts_manager.cc:194] Registered new tserver with Master: 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211)
I20260809 00:42:49.269963 9315 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:34145
I20260809 00:42:49.272056 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 9791
I20260809 00:42:49.272172 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:49.272399 31139 external_mini_cluster.cc:1428] Setting key 22bf9f632b78db41a0b07020d474aec0
I20260809 00:42:49.273386 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:42:49.276113 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.269898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41501 (local address 127.30.104.195:58263)
0809 00:42:49.270028 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.270039 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.270100 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.270212 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.270213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.270428 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:49.270509 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.270511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.271142 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.271142 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.271730 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.271732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.272080 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.272081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.272160 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.272763 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:49.272769 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:49.273313 (+ 544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:49.275056 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.275057 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.275057 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.275227 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.275481 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.275482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.275483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.275555 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.275848 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:49.275849 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:49.275896 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.275953 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.276035 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260809 00:42:49.276177 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.269969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:58263 (local address 127.30.104.254:41501)
0809 00:42:49.270017 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.270019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.270085 (+ 66us) server_negotiation.cc:407] Connection header received
0809 00:42:49.270156 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.270156 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.270166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.270210 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.270592 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.271050 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.271799 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.272004 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.272057 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.273390 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:49.273391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:49.273391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:49.273399 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:49.274962 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.275302 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.275303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.275303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.275369 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.275622 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.275623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.275623 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.275775 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:49.275845 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.276012 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.276100 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:42:49.299330 9386 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:49.293560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.30.104.254:41501)
0809 00:42:49.293634 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.293636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.293648 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:42:49.293684 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.293685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.293695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.293757 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.294354 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.294869 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.295719 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.295920 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.295997 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.299234 (+ 3237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46112: BlockingRecv error: recv got EOF from 127.0.0.1:46112 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236169, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:49.307878 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.300239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.30.104.254:41501)
0809 00:42:49.300331 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.300333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.300435 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:42:49.300551 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.300552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.300563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.300653 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.301145 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.301673 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.302368 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.302573 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.302651 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.304983 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:49.304984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:49.304985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:49.304995 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:49.306482 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.306844 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.306846 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.306846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.306904 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.307171 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.307172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.307173 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.307306 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:49.307529 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.307704 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.307785 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 00:42:49.310189 9315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:49.312487 9315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:49.321027 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.315996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:41977 (local address 127.0.0.1:38416)
0809 00:42:49.316374 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.316421 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.316503 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.317094 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.317097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.317100 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.317167 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.317169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.318364 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.318365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.319804 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.319807 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.320687 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.320688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.320771 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.320785 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.320868 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.320926 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":99,"threads_started":1}
I20260809 00:42:49.321043 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.316483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38416 (local address 127.30.104.194:41977)
0809 00:42:49.316880 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.316886 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.316896 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:49.316936 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.316938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.316953 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.317048 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.317280 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.318251 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.319917 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.320590 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.320706 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.320770 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.320895 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.320935 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":104,"threads_started":1}
I20260809 00:42:49.321427 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.316237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:43601 (local address 127.0.0.1:51828)
0809 00:42:49.317148 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.317159 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.317321 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.317545 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.317546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.317548 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.317589 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.317591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.318940 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.318941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.320243 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.320245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.321197 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.321198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.321256 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.321260 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.321305 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.321343 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":78,"thread_start_us":105,"threads_started":1}
I20260809 00:42:49.321592 9998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.316030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:36211 (local address 127.0.0.1:35342)
0809 00:42:49.316618 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.316628 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.316677 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.317299 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.317300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.317302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.317352 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.317354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.318938 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.318939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.320241 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.320245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.321320 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.321322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.321391 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.321396 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.321446 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.321498 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":61,"thread_start_us":103,"threads_started":1}
I20260809 00:42:49.321658 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.316419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51828 (local address 127.30.104.193:43601)
0809 00:42:49.316614 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.316618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.317245 (+ 627us) server_negotiation.cc:407] Connection header received
0809 00:42:49.317424 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.317427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.317443 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.317547 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.317647 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.318853 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.320349 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.321017 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.321101 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.321157 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.321520 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.321562 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
I20260809 00:42:49.322131 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.316163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.30.104.195:36211)
0809 00:42:49.316364 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.316370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.316679 (+ 309us) server_negotiation.cc:407] Connection header received
0809 00:42:49.316895 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.316898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.316919 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.317048 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.317515 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.318819 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.320338 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.321133 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.321230 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.321286 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.322024 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.322059 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":94,"threads_started":1}
I20260809 00:42:49.322345 9685 tablet_service.cc:1467] Processing CreateTablet for tablet 89737003ae554c2eb06f2470828eaba6 (DEFAULT_TABLE table=test-table [id=6ca14d52874a463b966a9ef695d34e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.322340 9501 tablet_service.cc:1467] Processing CreateTablet for tablet 89737003ae554c2eb06f2470828eaba6 (DEFAULT_TABLE table=test-table [id=6ca14d52874a463b966a9ef695d34e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.322753 9685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89737003ae554c2eb06f2470828eaba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.322753 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89737003ae554c2eb06f2470828eaba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.322829 9878 tablet_service.cc:1467] Processing CreateTablet for tablet 89737003ae554c2eb06f2470828eaba6 (DEFAULT_TABLE table=test-table [id=6ca14d52874a463b966a9ef695d34e2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.323113 9878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89737003ae554c2eb06f2470828eaba6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.326146 10003 tablet_bootstrap.cc:492] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Bootstrap starting.
I20260809 00:42:49.326786 10003 tablet_bootstrap.cc:654] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.326982 10004 tablet_bootstrap.cc:492] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Bootstrap starting.
I20260809 00:42:49.327178 10003 log.cc:824] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:49.327728 10004 tablet_bootstrap.cc:654] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.328102 10003 tablet_bootstrap.cc:492] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: No bootstrap required, opened a new log
I20260809 00:42:49.328148 10004 log.cc:824] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:49.328202 10003 ts_tablet_manager.cc:1397] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:42:49.328538 10005 tablet_bootstrap.cc:492] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Bootstrap starting.
I20260809 00:42:49.329237 10005 tablet_bootstrap.cc:654] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.329602 10005 log.cc:824] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:49.329887 10003 raft_consensus.cc:348] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.330056 10003 raft_consensus.cc:374] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.330125 10003 raft_consensus.cc:729] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d49785a987d455fbd83095328dc42ae, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.330248 10004 tablet_bootstrap.cc:492] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: No bootstrap required, opened a new log
I20260809 00:42:49.330322 10004 ts_tablet_manager.cc:1397] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:42:49.330279 10003 consensus_queue.cc:260] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.330626 10003 ts_tablet_manager.cc:1428] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:42:49.330668 9966 heartbeater.cc:503] Master 127.30.104.254:41501 was elected leader, sending a full tablet report...
I20260809 00:42:49.332659 10005 tablet_bootstrap.cc:492] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: No bootstrap required, opened a new log
I20260809 00:42:49.332751 10005 ts_tablet_manager.cc:1397] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 00:42:49.332737 10004 raft_consensus.cc:348] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.332949 10004 raft_consensus.cc:374] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.333002 10004 raft_consensus.cc:729] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4e537a618f446cb348d1061782125d, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.333120 10004 consensus_queue.cc:260] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.333400 10004 ts_tablet_manager.cc:1428] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:42:49.333467 9777 heartbeater.cc:503] Master 127.30.104.254:41501 was elected leader, sending a full tablet report...
I20260809 00:42:49.334784 10005 raft_consensus.cc:348] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.334920 10005 raft_consensus.cc:374] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.334942 10005 raft_consensus.cc:729] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d77fabd64134dcab7f8e2566ef79351, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.335059 10005 consensus_queue.cc:260] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.335307 10005 ts_tablet_manager.cc:1428] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:42:49.335371 9588 heartbeater.cc:503] Master 127.30.104.254:41501 was elected leader, sending a full tablet report...
W20260809 00:42:49.351194 9778 tablet.cc:2369] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:42:49.433663 9590 tablet.cc:2369] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:42:49.516521 9968 tablet.cc:2369] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:42:49.532315 10012 raft_consensus.cc:482] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:49.532539 10012 raft_consensus.cc:504] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.532966 10012 leader_election.cc:288] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211), 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601)
I20260809 00:42:49.536880 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.533231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:46949 (local address 127.30.104.195:36211)
0809 00:42:49.533280 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.533282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.533296 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:42:49.533359 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.533360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.533372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.533445 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.533623 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.534405 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.535879 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.536450 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.536536 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.536546 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.536790 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.536822 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:42:49.536970 9635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.533168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:36211 (local address 127.30.104.194:46949)
0809 00:42:49.533227 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.533241 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.533295 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.533454 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.533455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.533456 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.533533 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.533535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.534493 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.534494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.535781 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.535784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.536566 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.536568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.536698 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.536703 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.536780 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.536837 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":22}
I20260809 00:42:49.537225 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.533315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:60085 (local address 127.30.104.193:43601)
0809 00:42:49.533360 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.533362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.533500 (+ 138us) server_negotiation.cc:407] Connection header received
0809 00:42:49.533533 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.533534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.533548 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.533626 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.533777 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.534490 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.535880 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.536481 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.536545 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.536554 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.537091 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.537142 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:42:49.537384 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89737003ae554c2eb06f2470828eaba6" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d49785a987d455fbd83095328dc42ae" is_pre_election: true
I20260809 00:42:49.537514 10015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.533175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:43601 (local address 127.30.104.194:60085)
0809 00:42:49.533412 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.533422 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.533479 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.533650 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.533650 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.533652 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.533711 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.533712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.534561 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.534562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.535781 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.535784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.536584 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.536585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.536981 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.536986 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.537328 (+ 342us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.537420 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":29,"thread_start_us":130,"threads_started":1}
I20260809 00:42:49.537560 9903 raft_consensus.cc:2454] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 0.
I20260809 00:42:49.537806 9627 leader_election.cc:302] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d49785a987d455fbd83095328dc42ae, 5d4e537a618f446cb348d1061782125d; no voters:
I20260809 00:42:49.537976 10012 raft_consensus.cc:2781] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:49.538041 10012 raft_consensus.cc:482] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:49.537985 9529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89737003ae554c2eb06f2470828eaba6" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d77fabd64134dcab7f8e2566ef79351" is_pre_election: true
I20260809 00:42:49.538077 10012 raft_consensus.cc:3037] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.538154 9529 raft_consensus.cc:2454] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 0.
I20260809 00:42:49.538897 10012 raft_consensus.cc:504] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.539055 10012 leader_election.cc:288] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 election: Requested vote from peers 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211), 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601)
I20260809 00:42:49.539209 9529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89737003ae554c2eb06f2470828eaba6" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d77fabd64134dcab7f8e2566ef79351"
I20260809 00:42:49.539238 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89737003ae554c2eb06f2470828eaba6" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d49785a987d455fbd83095328dc42ae"
I20260809 00:42:49.539296 9529 raft_consensus.cc:3037] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.539304 9903 raft_consensus.cc:3037] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.540239 9529 raft_consensus.cc:2454] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 1.
I20260809 00:42:49.540239 9903 raft_consensus.cc:2454] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 1.
I20260809 00:42:49.540460 9623 leader_election.cc:302] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d77fabd64134dcab7f8e2566ef79351, 5d4e537a618f446cb348d1061782125d; no voters:
I20260809 00:42:49.540570 10012 raft_consensus.cc:2781] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:49.540732 10012 raft_consensus.cc:686] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 1 LEADER]: Becoming Leader. State: Replica: 5d4e537a618f446cb348d1061782125d, State: Running, Role: LEADER
I20260809 00:42:49.540843 10012 consensus_queue.cc:237] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } }
I20260809 00:42:49.541775 9315 catalog_manager.cc:5495] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4e537a618f446cb348d1061782125d (127.30.104.194). New cstate: current_term: 1 leader_uuid: "5d4e537a618f446cb348d1061782125d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:49.549602 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.547170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38428 (local address 127.30.104.194:41977)
0809 00:42:49.547233 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.547235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.547261 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:42:49.547290 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.547291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.547307 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.547360 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:49.547571 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.548044 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.548832 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.549034 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.549103 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.549282 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:49.549392 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:49.549468 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.549524 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.549543 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:42:49.551607 9903 raft_consensus.cc:1261] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Refusing update from remote peer 5d4e537a618f446cb348d1061782125d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:49.551609 9529 raft_consensus.cc:1261] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Refusing update from remote peer 5d4e537a618f446cb348d1061782125d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:49.552266 10012 consensus_queue.cc:1035] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:49.552477 10016 consensus_queue.cc:1035] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:49.555393 10029 mvcc.cc:205] Tried to move back new op lower bound from 7316423350479872000 to 7316423350439809024. Current Snapshot: MvccSnapshot[applied={T|T < 7316423350479872000}]
I20260809 00:42:49.557704 9315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:49.557821 9315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:49.558701 9315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: Sending DeleteTablet for 3 replicas of tablet 89737003ae554c2eb06f2470828eaba6
I20260809 00:42:49.559082 9878 tablet_service.cc:1514] Processing DeleteTablet for tablet 89737003ae554c2eb06f2470828eaba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:35342
I20260809 00:42:49.559106 9685 tablet_service.cc:1514] Processing DeleteTablet for tablet 89737003ae554c2eb06f2470828eaba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38416
I20260809 00:42:49.559162 9501 tablet_service.cc:1514] Processing DeleteTablet for tablet 89737003ae554c2eb06f2470828eaba6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:51828
I20260809 00:42:49.559243 9315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 00:42:49.559350 10042 tablet_replica.cc:330] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: stopping tablet replica
W20260809 00:42:49.559401 9315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:49.559474 10042 raft_consensus.cc:2229] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:49.559580 10042 raft_consensus.cc:2258] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.560223 10042 ts_tablet_manager.cc:1905] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.560690 10040 tablet_replica.cc:330] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: stopping tablet replica
I20260809 00:42:49.560860 10040 raft_consensus.cc:2229] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:49.561057 10040 raft_consensus.cc:2258] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.561061 9501 tablet_service.cc:1467] Processing CreateTablet for tablet fd6d42d0be2946dd99427e71681fa79f (DEFAULT_TABLE table=test-table [id=83a8005e00b94c9290c716770c98d17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.561184 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd6d42d0be2946dd99427e71681fa79f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.561574 10040 ts_tablet_manager.cc:1905] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.562093 10042 ts_tablet_manager.cc:1918] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:49.562160 10042 log.cc:1196] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/89737003ae554c2eb06f2470828eaba6
I20260809 00:42:49.562424 10042 ts_tablet_manager.cc:1939] T 89737003ae554c2eb06f2470828eaba6 P 0d77fabd64134dcab7f8e2566ef79351: Deleting consensus metadata
I20260809 00:42:49.562538 10005 tablet_bootstrap.cc:492] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: Bootstrap starting.
I20260809 00:42:49.562790 9292 catalog_manager.cc:4841] TS 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601): tablet 89737003ae554c2eb06f2470828eaba6 (table test-table [id=6ca14d52874a463b966a9ef695d34e2d]) successfully deleted
I20260809 00:42:49.562963 10005 tablet_bootstrap.cc:654] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.563054 9878 tablet_service.cc:1467] Processing CreateTablet for tablet fd6d42d0be2946dd99427e71681fa79f (DEFAULT_TABLE table=test-table [id=83a8005e00b94c9290c716770c98d17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.563174 9878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd6d42d0be2946dd99427e71681fa79f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.563350 10040 ts_tablet_manager.cc:1918] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:49.563474 10040 log.cc:1196] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/89737003ae554c2eb06f2470828eaba6
I20260809 00:42:49.563665 10005 tablet_bootstrap.cc:492] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: No bootstrap required, opened a new log
I20260809 00:42:49.563719 10005 ts_tablet_manager.cc:1397] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:49.563782 10040 ts_tablet_manager.cc:1939] T 89737003ae554c2eb06f2470828eaba6 P 2d49785a987d455fbd83095328dc42ae: Deleting consensus metadata
I20260809 00:42:49.563902 10005 raft_consensus.cc:348] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.563973 10005 raft_consensus.cc:374] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.563993 10005 raft_consensus.cc:729] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d77fabd64134dcab7f8e2566ef79351, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.564044 10005 consensus_queue.cc:260] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.564139 10005 ts_tablet_manager.cc:1428] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.564203 10041 tablet_replica.cc:330] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: stopping tablet replica
I20260809 00:42:49.564306 10041 raft_consensus.cc:2229] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:49.564453 10041 raft_consensus.cc:2258] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.564718 10003 tablet_bootstrap.cc:492] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: Bootstrap starting.
I20260809 00:42:49.565004 10041 ts_tablet_manager.cc:1905] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.565110 10003 tablet_bootstrap.cc:654] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.565596 9685 tablet_service.cc:1467] Processing CreateTablet for tablet fd6d42d0be2946dd99427e71681fa79f (DEFAULT_TABLE table=test-table [id=83a8005e00b94c9290c716770c98d17b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.565730 9685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd6d42d0be2946dd99427e71681fa79f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.566470 10041 ts_tablet_manager.cc:1918] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:49.566570 10041 log.cc:1196] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/89737003ae554c2eb06f2470828eaba6
I20260809 00:42:49.566856 10041 ts_tablet_manager.cc:1939] T 89737003ae554c2eb06f2470828eaba6 P 5d4e537a618f446cb348d1061782125d: Deleting consensus metadata
I20260809 00:42:49.567082 10004 tablet_bootstrap.cc:492] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: Bootstrap starting.
I20260809 00:42:49.567552 10004 tablet_bootstrap.cc:654] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.567637 9297 catalog_manager.cc:4841] TS 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211): tablet 89737003ae554c2eb06f2470828eaba6 (table test-table [id=6ca14d52874a463b966a9ef695d34e2d]) successfully deleted
I20260809 00:42:49.567637 9292 catalog_manager.cc:4841] TS 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977): tablet 89737003ae554c2eb06f2470828eaba6 (table test-table [id=6ca14d52874a463b966a9ef695d34e2d]) successfully deleted
I20260809 00:42:49.568043 10003 tablet_bootstrap.cc:492] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: No bootstrap required, opened a new log
I20260809 00:42:49.568099 10003 ts_tablet_manager.cc:1397] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 00:42:49.568225 10004 tablet_bootstrap.cc:492] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: No bootstrap required, opened a new log
I20260809 00:42:49.568266 10004 ts_tablet_manager.cc:1397] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:49.568280 10003 raft_consensus.cc:348] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.568336 10003 raft_consensus.cc:374] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.568348 10003 raft_consensus.cc:729] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d49785a987d455fbd83095328dc42ae, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.568413 10003 consensus_queue.cc:260] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.568421 10004 raft_consensus.cc:348] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.568522 10004 raft_consensus.cc:374] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.568521 10003 ts_tablet_manager.cc:1428] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.568547 10004 raft_consensus.cc:729] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4e537a618f446cb348d1061782125d, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.568599 10004 consensus_queue.cc:260] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.568686 10004 ts_tablet_manager.cc:1428] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 00:42:49.606271 10016 raft_consensus.cc:482] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:49.606410 10016 raft_consensus.cc:504] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.606609 10016 leader_election.cc:288] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601), 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211)
I20260809 00:42:49.606897 9529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd6d42d0be2946dd99427e71681fa79f" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d77fabd64134dcab7f8e2566ef79351" is_pre_election: true
I20260809 00:42:49.606897 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd6d42d0be2946dd99427e71681fa79f" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d49785a987d455fbd83095328dc42ae" is_pre_election: true
I20260809 00:42:49.607018 9903 raft_consensus.cc:2454] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 0.
I20260809 00:42:49.607018 9529 raft_consensus.cc:2454] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 0.
I20260809 00:42:49.607203 9623 leader_election.cc:302] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d77fabd64134dcab7f8e2566ef79351, 5d4e537a618f446cb348d1061782125d; no voters:
I20260809 00:42:49.607378 10016 raft_consensus.cc:2781] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:49.607422 10016 raft_consensus.cc:482] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:49.607501 10016 raft_consensus.cc:3037] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.608526 10016 raft_consensus.cc:504] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.608695 10016 leader_election.cc:288] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 election: Requested vote from peers 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601), 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211)
I20260809 00:42:49.608857 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd6d42d0be2946dd99427e71681fa79f" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d49785a987d455fbd83095328dc42ae"
I20260809 00:42:49.608875 9529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd6d42d0be2946dd99427e71681fa79f" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d77fabd64134dcab7f8e2566ef79351"
I20260809 00:42:49.608954 9903 raft_consensus.cc:3037] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.608976 9529 raft_consensus.cc:3037] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.609774 9903 raft_consensus.cc:2454] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 1.
I20260809 00:42:49.609994 9529 raft_consensus.cc:2454] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 1.
I20260809 00:42:49.610008 9627 leader_election.cc:302] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d49785a987d455fbd83095328dc42ae, 5d4e537a618f446cb348d1061782125d; no voters:
I20260809 00:42:49.610086 10016 raft_consensus.cc:2781] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:49.610131 10016 raft_consensus.cc:686] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 1 LEADER]: Becoming Leader. State: Replica: 5d4e537a618f446cb348d1061782125d, State: Running, Role: LEADER
I20260809 00:42:49.610177 10016 consensus_queue.cc:237] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.611092 9316 catalog_manager.cc:5495] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4e537a618f446cb348d1061782125d (127.30.104.194). New cstate: current_term: 1 leader_uuid: "5d4e537a618f446cb348d1061782125d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } health_report { overall_health: UNKNOWN } } }
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236168, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.254@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:49.653177 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.644264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41501 (local address 127.0.0.1:46122)
0809 00:42:49.644474 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.644484 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.644543 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.644709 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.644709 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.645020 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:49.645078 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.645079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.646144 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.646145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.646703 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.646705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.647240 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.647241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.647295 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.648106 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:49.648111 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:49.649785 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:49.651609 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.651619 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.651622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.652068 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.652332 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:49.652333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:49.652333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:49.652423 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:49.652725 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:49.652733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:49.652793 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.652876 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.653023 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260809 00:42:49.653178 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.644383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.30.104.254:41501)
0809 00:42:49.644445 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.644448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.644562 (+ 114us) server_negotiation.cc:407] Connection header received
0809 00:42:49.644598 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.644599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.644612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.644695 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.645183 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.646024 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.646817 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.647132 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.647222 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.649889 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:49.649891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:49.649891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:49.649916 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:49.651514 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.652175 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.652176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.652176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.652244 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.652502 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.652503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.652503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.652632 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:49.652718 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.652924 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.653080 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":33}
I20260809 00:42:49.656041 9316 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:46122:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 00:42:49.656301 9316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:49.658036 9878 tablet_service.cc:1467] Processing CreateTablet for tablet 57cb34a14f1f4be28dfe88ccd9df3fa6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6de80385b8b0494ea1a7d0400bec1f83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:49.658109 9685 tablet_service.cc:1467] Processing CreateTablet for tablet 57cb34a14f1f4be28dfe88ccd9df3fa6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6de80385b8b0494ea1a7d0400bec1f83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:49.658150 9501 tablet_service.cc:1467] Processing CreateTablet for tablet 57cb34a14f1f4be28dfe88ccd9df3fa6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6de80385b8b0494ea1a7d0400bec1f83]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 00:42:49.658234 9878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57cb34a14f1f4be28dfe88ccd9df3fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.658241 9685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57cb34a14f1f4be28dfe88ccd9df3fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.658319 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57cb34a14f1f4be28dfe88ccd9df3fa6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.659973 10003 tablet_bootstrap.cc:492] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: Bootstrap starting.
I20260809 00:42:49.659983 10004 tablet_bootstrap.cc:492] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: Bootstrap starting.
I20260809 00:42:49.660399 10003 tablet_bootstrap.cc:654] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.660445 10004 tablet_bootstrap.cc:654] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.660796 10005 tablet_bootstrap.cc:492] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: Bootstrap starting.
I20260809 00:42:49.661142 10003 tablet_bootstrap.cc:492] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: No bootstrap required, opened a new log
I20260809 00:42:49.661209 10005 tablet_bootstrap.cc:654] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.661221 10003 ts_tablet_manager.cc:1397] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:49.661407 10003 raft_consensus.cc:348] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.661509 10003 raft_consensus.cc:374] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.661548 10003 raft_consensus.cc:729] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d49785a987d455fbd83095328dc42ae, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.661597 10003 consensus_queue.cc:260] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.661715 10003 ts_tablet_manager.cc:1428] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 00:42:49.661867 10011 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.661943 10005 tablet_bootstrap.cc:492] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: No bootstrap required, opened a new log
I20260809 00:42:49.661969 10011 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:49.661990 10005 ts_tablet_manager.cc:1397] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:49.662150 10004 tablet_bootstrap.cc:492] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: No bootstrap required, opened a new log
I20260809 00:42:49.662137 10005 raft_consensus.cc:348] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.662195 10004 ts_tablet_manager.cc:1397] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:42:49.662217 10005 raft_consensus.cc:374] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.662243 10005 raft_consensus.cc:729] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d77fabd64134dcab7f8e2566ef79351, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.662295 10005 consensus_queue.cc:260] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.662340 10004 raft_consensus.cc:348] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.662397 10005 ts_tablet_manager.cc:1428] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.662408 10004 raft_consensus.cc:374] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.662444 10004 raft_consensus.cc:729] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4e537a618f446cb348d1061782125d, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.662487 10004 consensus_queue.cc:260] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.662524 10013 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.662585 10004 ts_tablet_manager.cc:1428] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.662609 10013 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:49.662703 10016 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.662770 10016 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:49.814976 10011 raft_consensus.cc:482] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:49.815112 10011 raft_consensus.cc:504] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.815582 10011 leader_election.cc:288] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601), 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977)
I20260809 00:42:49.818686 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.816070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:57431 (local address 127.30.104.194:41977)
0809 00:42:49.816114 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.816116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.816195 (+ 79us) server_negotiation.cc:407] Connection header received
0809 00:42:49.816224 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.816225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.816236 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.816295 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.816488 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.816926 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.817940 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.818328 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.818413 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.818422 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.818567 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.818610 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:42:49.818693 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.815845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:35631 (local address 127.30.104.193:43601)
0809 00:42:49.815909 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.815912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.816050 (+ 138us) server_negotiation.cc:407] Connection header received
0809 00:42:49.816111 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.816113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.816125 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.816182 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.816369 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.816899 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.817983 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.818325 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.818414 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.818422 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.818590 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.818626 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:42:49.818712 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.815786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:43601 (local address 127.30.104.195:35631)
0809 00:42:49.815947 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.815958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.816041 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.816202 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.816203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.816206 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.816296 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.816298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.816997 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.816998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.817836 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.817838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.818413 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.818414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.818490 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.818494 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.818571 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.818612 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":29,"thread_start_us":104,"threads_started":1}
I20260809 00:42:49.818714 10059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.815982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:41977 (local address 127.30.104.195:57431)
0809 00:42:49.816106 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.816117 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.816174 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.816337 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.816338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.816342 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.816407 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.816409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.817033 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.817034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.817839 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.817841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.818422 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.818424 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.818492 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.818495 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.818538 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.818602 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":33,"thread_start_us":56,"threads_started":1}
I20260809 00:42:49.818970 9529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57cb34a14f1f4be28dfe88ccd9df3fa6" candidate_uuid: "2d49785a987d455fbd83095328dc42ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d77fabd64134dcab7f8e2566ef79351" is_pre_election: true
I20260809 00:42:49.819062 9529 raft_consensus.cc:2454] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d49785a987d455fbd83095328dc42ae in term 0.
I20260809 00:42:49.819237 9813 leader_election.cc:302] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d77fabd64134dcab7f8e2566ef79351, 2d49785a987d455fbd83095328dc42ae; no voters:
I20260809 00:42:49.819406 10011 raft_consensus.cc:2781] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:49.819496 10011 raft_consensus.cc:482] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:49.819535 10011 raft_consensus.cc:3037] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.819523 9715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57cb34a14f1f4be28dfe88ccd9df3fa6" candidate_uuid: "2d49785a987d455fbd83095328dc42ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4e537a618f446cb348d1061782125d" is_pre_election: true
I20260809 00:42:49.819680 9715 raft_consensus.cc:2454] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d49785a987d455fbd83095328dc42ae in term 0.
I20260809 00:42:49.820418 10011 raft_consensus.cc:504] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.820568 10011 leader_election.cc:288] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [CANDIDATE]: Term 1 election: Requested vote from peers 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601), 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977)
I20260809 00:42:49.820806 9715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57cb34a14f1f4be28dfe88ccd9df3fa6" candidate_uuid: "2d49785a987d455fbd83095328dc42ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4e537a618f446cb348d1061782125d"
I20260809 00:42:49.820806 9529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57cb34a14f1f4be28dfe88ccd9df3fa6" candidate_uuid: "2d49785a987d455fbd83095328dc42ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d77fabd64134dcab7f8e2566ef79351"
I20260809 00:42:49.820892 9529 raft_consensus.cc:3037] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.820892 9715 raft_consensus.cc:3037] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.821707 9529 raft_consensus.cc:2454] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d49785a987d455fbd83095328dc42ae in term 1.
I20260809 00:42:49.821836 9715 raft_consensus.cc:2454] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d49785a987d455fbd83095328dc42ae in term 1.
I20260809 00:42:49.821938 9813 leader_election.cc:302] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d77fabd64134dcab7f8e2566ef79351, 2d49785a987d455fbd83095328dc42ae; no voters:
I20260809 00:42:49.822041 10011 raft_consensus.cc:2781] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:49.822172 10011 raft_consensus.cc:686] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [term 1 LEADER]: Becoming Leader. State: Replica: 2d49785a987d455fbd83095328dc42ae, State: Running, Role: LEADER
I20260809 00:42:49.822252 10011 consensus_queue.cc:237] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.822618 10060 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: TxnStatusTablet state changed. Reason: New leader 2d49785a987d455fbd83095328dc42ae. Latest consensus state: current_term: 1 leader_uuid: "2d49785a987d455fbd83095328dc42ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.822796 10060 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:49.823026 10064 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 00:42:49.823145 10064 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 00:42:49.823033 9316 catalog_manager.cc:5495] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d49785a987d455fbd83095328dc42ae (127.30.104.195). New cstate: current_term: 1 leader_uuid: "2d49785a987d455fbd83095328dc42ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } health_report { overall_health: HEALTHY } } }
I20260809 00:42:49.850585 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.847651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:36211 (local address 127.0.0.1:35348)
0809 00:42:49.847739 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.847753 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.847811 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.848040 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.848041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.848042 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:49.848101 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.848104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.848782 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.848783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.849402 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.849404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.849944 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.849945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.850020 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.850047 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:49.850379 (+ 332us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:49.850392 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.850451 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.850510 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 00:42:49.850567 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.847741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35348 (local address 127.30.104.195:36211)
0809 00:42:49.847798 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.847800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.847811 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:49.847920 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.847922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.847944 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.848035 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:49.848177 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.848682 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.849538 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.849843 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.849941 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.850158 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:49.850280 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:49.850393 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.850453 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.850479 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:42:49.852181 9715 raft_consensus.cc:1261] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Refusing update from remote peer 2d49785a987d455fbd83095328dc42ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:49.852182 9529 raft_consensus.cc:1261] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Refusing update from remote peer 2d49785a987d455fbd83095328dc42ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:49.852483 10011 consensus_queue.cc:1035] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:49.852626 10060 consensus_queue.cc:1035] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:49.853307 10013 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: TxnStatusTablet state changed. Reason: New leader 2d49785a987d455fbd83095328dc42ae. Latest consensus state: current_term: 1 leader_uuid: "2d49785a987d455fbd83095328dc42ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.853421 10013 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:49.853511 10016 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: TxnStatusTablet state changed. Reason: New leader 2d49785a987d455fbd83095328dc42ae. Latest consensus state: current_term: 1 leader_uuid: "2d49785a987d455fbd83095328dc42ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.853608 10016 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:49.854235 10011 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d49785a987d455fbd83095328dc42ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.854318 10011 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:49.854417 10011 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d49785a987d455fbd83095328dc42ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.854483 10011 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 2d49785a987d455fbd83095328dc42ae: This TxnStatusTablet replica's current role is: LEADER
I20260809 00:42:49.854703 10027 mvcc.cc:205] Tried to move back new op lower bound from 7316423351711932416 to 7316423351592747008. Current Snapshot: MvccSnapshot[applied={T|T < 7316423351711932416}]
I20260809 00:42:49.854902 10013 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d49785a987d455fbd83095328dc42ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.855005 10013 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 0d77fabd64134dcab7f8e2566ef79351: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:49.855684 10026 mvcc.cc:205] Tried to move back new op lower bound from 7316423351711932416 to 7316423351592747008. Current Snapshot: MvccSnapshot[applied={T|T < 7316423351711932416}]
I20260809 00:42:49.855777 10016 tablet_replica.cc:439] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d49785a987d455fbd83095328dc42ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } } }
I20260809 00:42:49.855845 10016 tablet_replica.cc:442] T 57cb34a14f1f4be28dfe88ccd9df3fa6 P 5d4e537a618f446cb348d1061782125d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 00:42:49.861562 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.858795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:45819 (local address 127.30.104.195:36211)
0809 00:42:49.858862 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.858864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.858960 (+ 96us) server_negotiation.cc:407] Connection header received
0809 00:42:49.859008 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.859008 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.859018 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.859062 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:49.859190 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.859771 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.860894 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.861095 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.861154 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.861347 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:49.861374 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:49.861418 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.861484 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.861503 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:49.861631 10073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.858693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:36211 (local address 127.30.104.194:45819)
0809 00:42:49.858881 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.858893 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.858956 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.859074 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.859075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.859076 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:49.859130 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.859131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.859849 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.859850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.860796 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.860798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.861180 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.861181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.861242 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.861259 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:49.861434 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:49.861436 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.861484 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.861535 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260809 00:42:49.864037 9903 raft_consensus.cc:1261] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Refusing update from remote peer 5d4e537a618f446cb348d1061782125d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:49.864064 9528 raft_consensus.cc:1261] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Refusing update from remote peer 5d4e537a618f446cb348d1061782125d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:49.864343 10032 consensus_queue.cc:1035] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:49.864478 10032 consensus_queue.cc:1035] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:49.873822 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.870428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:37985 (local address 127.30.104.194:41977)
0809 00:42:49.870470 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.870473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.870753 (+ 280us) server_negotiation.cc:407] Connection header received
0809 00:42:49.870894 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.870895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.870906 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.870936 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:49.871169 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.871731 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.873053 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.873244 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.873316 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.873526 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:49.873557 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:49.873676 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.873739 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.873761 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:42:49.873889 10078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.870385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:41977 (local address 127.30.104.195:37985)
0809 00:42:49.870628 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.870692 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.870742 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.871003 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.871004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.871006 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 00:42:49.871094 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.871097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.871812 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.871813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.872946 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.872950 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.873357 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.873359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.873425 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.873443 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 00:42:49.873685 (+ 242us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 00:42:49.873687 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.873735 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.873783 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
I20260809 00:42:49.882462 9315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:49.882565 9315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:49.883388 9315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: Sending DeleteTablet for 3 replicas of tablet fd6d42d0be2946dd99427e71681fa79f
I20260809 00:42:49.883697 9501 tablet_service.cc:1514] Processing DeleteTablet for tablet fd6d42d0be2946dd99427e71681fa79f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:51828
I20260809 00:42:49.883819 10042 tablet_replica.cc:330] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: stopping tablet replica
I20260809 00:42:49.883903 10042 raft_consensus.cc:2229] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:49.883900 9685 tablet_service.cc:1514] Processing DeleteTablet for tablet fd6d42d0be2946dd99427e71681fa79f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38416
I20260809 00:42:49.883947 10042 raft_consensus.cc:2258] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.883997 10041 tablet_replica.cc:330] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: stopping tablet replica
I20260809 00:42:49.884053 10041 raft_consensus.cc:2229] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:49.884106 10041 raft_consensus.cc:2258] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.884274 10042 ts_tablet_manager.cc:1905] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.884462 9878 tablet_service.cc:1514] Processing DeleteTablet for tablet fd6d42d0be2946dd99427e71681fa79f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:35342
I20260809 00:42:49.884601 10041 ts_tablet_manager.cc:1905] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.885808 10040 tablet_replica.cc:330] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: stopping tablet replica
I20260809 00:42:49.885923 10040 raft_consensus.cc:2229] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:49.885970 10040 raft_consensus.cc:2258] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.886201 10042 ts_tablet_manager.cc:1918] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:49.886260 10042 log.cc:1196] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/fd6d42d0be2946dd99427e71681fa79f
I20260809 00:42:49.886325 10040 ts_tablet_manager.cc:1905] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.886505 10042 ts_tablet_manager.cc:1939] T fd6d42d0be2946dd99427e71681fa79f P 0d77fabd64134dcab7f8e2566ef79351: Deleting consensus metadata
I20260809 00:42:49.886870 9292 catalog_manager.cc:4841] TS 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601): tablet fd6d42d0be2946dd99427e71681fa79f (table test-table [id=83a8005e00b94c9290c716770c98d17b]) successfully deleted
I20260809 00:42:49.887735 10041 ts_tablet_manager.cc:1918] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:49.887794 10041 log.cc:1196] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/fd6d42d0be2946dd99427e71681fa79f
I20260809 00:42:49.888023 10041 ts_tablet_manager.cc:1939] T fd6d42d0be2946dd99427e71681fa79f P 5d4e537a618f446cb348d1061782125d: Deleting consensus metadata
I20260809 00:42:49.888170 10040 ts_tablet_manager.cc:1918] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:49.888223 10040 log.cc:1196] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/fd6d42d0be2946dd99427e71681fa79f
I20260809 00:42:49.888360 9292 catalog_manager.cc:4841] TS 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977): tablet fd6d42d0be2946dd99427e71681fa79f (table test-table [id=83a8005e00b94c9290c716770c98d17b]) successfully deleted
I20260809 00:42:49.888440 10040 ts_tablet_manager.cc:1939] T fd6d42d0be2946dd99427e71681fa79f P 2d49785a987d455fbd83095328dc42ae: Deleting consensus metadata
I20260809 00:42:49.888784 9297 catalog_manager.cc:4841] TS 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211): tablet fd6d42d0be2946dd99427e71681fa79f (table test-table [id=83a8005e00b94c9290c716770c98d17b]) successfully deleted
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:42:49.909201 9386 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:49.903011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.30.104.254:41501)
0809 00:42:49.903081 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.903083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.903200 (+ 117us) server_negotiation.cc:407] Connection header received
0809 00:42:49.903241 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.903242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.903254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.903333 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.903988 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.904713 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.905537 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.905819 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.905915 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.909130 (+ 3215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46130: BlockingRecv error: recv got EOF from 127.0.0.1:46130 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 00:42:49 dist-test-slave-c12x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM
I20260809 00:42:49.918767 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.910297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46134 (local address 127.30.104.254:41501)
0809 00:42:49.910345 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.910347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.910457 (+ 110us) server_negotiation.cc:407] Connection header received
0809 00:42:49.910527 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.910527 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.910536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.910585 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:49.911259 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.911851 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.912852 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.913048 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.913105 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.915957 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:49.915959 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:49.915959 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:49.915974 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:49.917546 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.917904 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.917905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.917906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.917967 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:49.918179 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:49.918180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:49.918180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:49.918346 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:49.918452 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.918564 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.918651 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:42:49.920506 9315 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:49.920660 9315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:49.922171 9685 tablet_service.cc:1467] Processing CreateTablet for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 (DEFAULT_TABLE table=test-table [id=8bc0b21941c547668fa615146c7e68b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.922171 9501 tablet_service.cc:1467] Processing CreateTablet for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 (DEFAULT_TABLE table=test-table [id=8bc0b21941c547668fa615146c7e68b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.922322 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.922322 9685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.923785 10004 tablet_bootstrap.cc:492] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: Bootstrap starting.
I20260809 00:42:49.923832 9878 tablet_service.cc:1467] Processing CreateTablet for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 (DEFAULT_TABLE table=test-table [id=8bc0b21941c547668fa615146c7e68b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.923949 9878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.924165 10004 tablet_bootstrap.cc:654] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.924839 10004 tablet_bootstrap.cc:492] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: No bootstrap required, opened a new log
I20260809 00:42:49.924896 10004 ts_tablet_manager.cc:1397] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:49.925009 10004 raft_consensus.cc:348] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.925055 10003 tablet_bootstrap.cc:492] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: Bootstrap starting.
I20260809 00:42:49.925077 10004 raft_consensus.cc:374] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.925099 10004 raft_consensus.cc:729] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4e537a618f446cb348d1061782125d, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.925129 10005 tablet_bootstrap.cc:492] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: Bootstrap starting.
I20260809 00:42:49.925148 10004 consensus_queue.cc:260] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.925235 10004 ts_tablet_manager.cc:1428] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.925505 10003 tablet_bootstrap.cc:654] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.925593 10005 tablet_bootstrap.cc:654] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.926268 10003 tablet_bootstrap.cc:492] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: No bootstrap required, opened a new log
I20260809 00:42:49.926314 10003 ts_tablet_manager.cc:1397] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:49.926486 10003 raft_consensus.cc:348] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.926563 10003 raft_consensus.cc:374] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.926580 10003 raft_consensus.cc:729] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d49785a987d455fbd83095328dc42ae, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.926616 10003 consensus_queue.cc:260] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.926700 10003 ts_tablet_manager.cc:1428] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.927264 10005 tablet_bootstrap.cc:492] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: No bootstrap required, opened a new log
I20260809 00:42:49.927350 10005 ts_tablet_manager.cc:1397] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:42:49.927634 10005 raft_consensus.cc:348] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.927721 10005 raft_consensus.cc:374] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.927742 10005 raft_consensus.cc:729] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d77fabd64134dcab7f8e2566ef79351, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.927784 10005 consensus_queue.cc:260] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.927891 10005 ts_tablet_manager.cc:1428] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:42:49.932677 10013 raft_consensus.cc:482] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:49.932755 10013 raft_consensus.cc:504] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.933063 10013 leader_election.cc:288] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977), 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211)
I20260809 00:42:49.937817 10104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.933245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:41977 (local address 127.30.104.193:37513)
0809 00:42:49.933388 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.933403 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.933463 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.933610 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.933610 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.933612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.933680 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.933682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.934772 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.934773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.936048 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.936050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.937561 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.937561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.937628 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.937639 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.937696 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.937733 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260809 00:42:49.937835 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.933432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:37513 (local address 127.30.104.194:41977)
0809 00:42:49.933481 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.933483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.933492 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:49.933520 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.933521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.933534 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.933604 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.933768 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.934548 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.936185 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.936750 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.936849 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.936859 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.937719 (+ 860us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.937766 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:42:49.938382 9715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ca2daeda3d45c3a2d0701b08ccbbb5" candidate_uuid: "0d77fabd64134dcab7f8e2566ef79351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4e537a618f446cb348d1061782125d" is_pre_election: true
I20260809 00:42:49.938488 9715 raft_consensus.cc:2454] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d77fabd64134dcab7f8e2566ef79351 in term 0.
I20260809 00:42:49.938714 9433 leader_election.cc:302] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d77fabd64134dcab7f8e2566ef79351, 5d4e537a618f446cb348d1061782125d; no voters:
I20260809 00:42:49.938841 10013 raft_consensus.cc:2781] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:49.938889 10013 raft_consensus.cc:482] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:49.938915 10013 raft_consensus.cc:3037] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.939576 9997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.933957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:36877 (local address 127.30.104.195:36211)
0809 00:42:49.934000 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.934002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.936517 (+ 2515us) server_negotiation.cc:407] Connection header received
0809 00:42:49.936582 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.936583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.936593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.936649 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:42:49.936815 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.937432 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.938720 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.939178 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.939264 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.939272 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.939464 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.939502 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:42:49.939613 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.933763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:36211 (local address 127.30.104.193:36877)
0809 00:42:49.936405 (+ 2642us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:49.936415 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:49.936526 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:49.936652 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:49.936652 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:49.936665 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:42:49.936739 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.936740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.937516 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.937517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.938633 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:49.938636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.939264 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.939266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:49.939337 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.939342 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:42:49.939477 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 00:42:49.939518 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2627,"thread_start_us":94,"threads_started":1}
I20260809 00:42:49.939790 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ca2daeda3d45c3a2d0701b08ccbbb5" candidate_uuid: "0d77fabd64134dcab7f8e2566ef79351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d49785a987d455fbd83095328dc42ae" is_pre_election: true
I20260809 00:42:49.939865 9903 raft_consensus.cc:2454] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d77fabd64134dcab7f8e2566ef79351 in term 0.
I20260809 00:42:49.939851 10013 raft_consensus.cc:504] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.940007 10013 leader_election.cc:288] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [CANDIDATE]: Term 1 election: Requested vote from peers 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977), 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211)
I20260809 00:42:49.940168 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ca2daeda3d45c3a2d0701b08ccbbb5" candidate_uuid: "0d77fabd64134dcab7f8e2566ef79351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d49785a987d455fbd83095328dc42ae"
I20260809 00:42:49.940234 9903 raft_consensus.cc:3037] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.940389 9715 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4ca2daeda3d45c3a2d0701b08ccbbb5" candidate_uuid: "0d77fabd64134dcab7f8e2566ef79351" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d4e537a618f446cb348d1061782125d"
I20260809 00:42:49.940461 9715 raft_consensus.cc:3037] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.940886 9903 raft_consensus.cc:2454] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d77fabd64134dcab7f8e2566ef79351 in term 1.
I20260809 00:42:49.941105 9438 leader_election.cc:302] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d77fabd64134dcab7f8e2566ef79351, 2d49785a987d455fbd83095328dc42ae; no voters:
I20260809 00:42:49.941193 10013 raft_consensus.cc:2781] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:49.941370 9715 raft_consensus.cc:2454] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d77fabd64134dcab7f8e2566ef79351 in term 1.
I20260809 00:42:49.941408 10013 raft_consensus.cc:686] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 LEADER]: Becoming Leader. State: Replica: 0d77fabd64134dcab7f8e2566ef79351, State: Running, Role: LEADER
I20260809 00:42:49.941484 10013 consensus_queue.cc:237] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.942266 9315 catalog_manager.cc:5495] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193). New cstate: current_term: 1 leader_uuid: "0d77fabd64134dcab7f8e2566ef79351" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:49.950901 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.948132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51842 (local address 127.30.104.193:43601)
0809 00:42:49.948197 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.948199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.948210 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:49.948296 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.948297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.948328 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.948428 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:49.948593 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.949107 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.949987 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:49.950201 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:49.950294 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:49.950485 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:49.950630 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:49.950724 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:49.950772 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:42:49.950797 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:42:49.952077 9903 raft_consensus.cc:1261] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Refusing update from remote peer 0d77fabd64134dcab7f8e2566ef79351: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:49.952152 9715 raft_consensus.cc:1261] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Refusing update from remote peer 0d77fabd64134dcab7f8e2566ef79351: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:49.952410 10107 consensus_queue.cc:1035] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:49.952574 10013 consensus_queue.cc:1035] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:49.957010 9314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:49.957124 9314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:49.957908 9314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: Sending DeleteTablet for 3 replicas of tablet c4ca2daeda3d45c3a2d0701b08ccbbb5
I20260809 00:42:49.958222 9501 tablet_service.cc:1514] Processing DeleteTablet for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:51828
I20260809 00:42:49.958238 9685 tablet_service.cc:1514] Processing DeleteTablet for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38416
I20260809 00:42:49.958341 10041 tablet_replica.cc:330] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: stopping tablet replica
I20260809 00:42:49.958344 10042 tablet_replica.cc:330] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: stopping tablet replica
I20260809 00:42:49.958396 10041 raft_consensus.cc:2229] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:49.958413 10042 raft_consensus.cc:2229] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:49.958410 9878 tablet_service.cc:1514] Processing DeleteTablet for tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:49 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:35342
I20260809 00:42:49.958446 10041 raft_consensus.cc:2258] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.958496 10042 raft_consensus.cc:2258] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.958504 10040 tablet_replica.cc:330] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: stopping tablet replica
I20260809 00:42:49.958561 10040 raft_consensus.cc:2229] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:49.958621 10040 raft_consensus.cc:2258] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:49.958618 9314 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:42:49.958770 9314 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:42:49.958858 10042 ts_tablet_manager.cc:1905] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.959019 10041 ts_tablet_manager.cc:1905] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.960180 9501 tablet_service.cc:1467] Processing CreateTablet for tablet 8830552390b04913971e3d59527ca56c (DEFAULT_TABLE table=test-table [id=a7a2ed7b6b2b4c43af90802c5a588e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.960289 9501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8830552390b04913971e3d59527ca56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.960609 10042 ts_tablet_manager.cc:1918] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:49.960660 10042 log.cc:1196] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/c4ca2daeda3d45c3a2d0701b08ccbbb5
I20260809 00:42:49.960777 10041 ts_tablet_manager.cc:1918] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:49.960840 10041 log.cc:1196] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/c4ca2daeda3d45c3a2d0701b08ccbbb5
I20260809 00:42:49.960884 10042 ts_tablet_manager.cc:1939] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 0d77fabd64134dcab7f8e2566ef79351: Deleting consensus metadata
I20260809 00:42:49.961122 10041 ts_tablet_manager.cc:1939] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 5d4e537a618f446cb348d1061782125d: Deleting consensus metadata
I20260809 00:42:49.961510 10005 tablet_bootstrap.cc:492] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: Bootstrap starting.
I20260809 00:42:49.961534 10040 ts_tablet_manager.cc:1905] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:49.961925 10005 tablet_bootstrap.cc:654] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.962399 9685 tablet_service.cc:1467] Processing CreateTablet for tablet 8830552390b04913971e3d59527ca56c (DEFAULT_TABLE table=test-table [id=a7a2ed7b6b2b4c43af90802c5a588e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.962507 9685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8830552390b04913971e3d59527ca56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.963079 10040 ts_tablet_manager.cc:1918] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:42:49.963143 10040 log.cc:1196] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/c4ca2daeda3d45c3a2d0701b08ccbbb5
I20260809 00:42:49.963395 10040 ts_tablet_manager.cc:1939] T c4ca2daeda3d45c3a2d0701b08ccbbb5 P 2d49785a987d455fbd83095328dc42ae: Deleting consensus metadata
I20260809 00:42:49.963747 10004 tablet_bootstrap.cc:492] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: Bootstrap starting.
I20260809 00:42:49.964071 10004 tablet_bootstrap.cc:654] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.964483 9292 catalog_manager.cc:4841] TS 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601): tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 (table test-table [id=8bc0b21941c547668fa615146c7e68b1]) successfully deleted
I20260809 00:42:49.964594 9292 catalog_manager.cc:4841] TS 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977): tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 (table test-table [id=8bc0b21941c547668fa615146c7e68b1]) successfully deleted
I20260809 00:42:49.964691 10004 tablet_bootstrap.cc:492] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: No bootstrap required, opened a new log
I20260809 00:42:49.964744 10004 ts_tablet_manager.cc:1397] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:49.964876 10004 raft_consensus.cc:348] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.964972 10004 raft_consensus.cc:374] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.964998 10004 raft_consensus.cc:729] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d4e537a618f446cb348d1061782125d, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.965045 10004 consensus_queue.cc:260] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.965162 10004 ts_tablet_manager.cc:1428] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.965732 9297 catalog_manager.cc:4841] TS 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211): tablet c4ca2daeda3d45c3a2d0701b08ccbbb5 (table test-table [id=8bc0b21941c547668fa615146c7e68b1]) successfully deleted
I20260809 00:42:49.965934 9878 tablet_service.cc:1467] Processing CreateTablet for tablet 8830552390b04913971e3d59527ca56c (DEFAULT_TABLE table=test-table [id=a7a2ed7b6b2b4c43af90802c5a588e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:42:49.966059 9878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8830552390b04913971e3d59527ca56c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:49.967410 10003 tablet_bootstrap.cc:492] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: Bootstrap starting.
I20260809 00:42:49.967851 10003 tablet_bootstrap.cc:654] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:49.968719 10003 tablet_bootstrap.cc:492] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: No bootstrap required, opened a new log
I20260809 00:42:49.968770 10003 ts_tablet_manager.cc:1397] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:49.968923 10003 raft_consensus.cc:348] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.968998 10003 raft_consensus.cc:374] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.969019 10003 raft_consensus.cc:729] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d49785a987d455fbd83095328dc42ae, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.969064 10003 consensus_queue.cc:260] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.969182 10003 ts_tablet_manager.cc:1428] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.971951 10005 tablet_bootstrap.cc:492] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: No bootstrap required, opened a new log
I20260809 00:42:49.972014 10005 ts_tablet_manager.cc:1397] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260809 00:42:49.972172 10005 raft_consensus.cc:348] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.972255 10005 raft_consensus.cc:374] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:49.972275 10005 raft_consensus.cc:729] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d77fabd64134dcab7f8e2566ef79351, State: Initialized, Role: FOLLOWER
I20260809 00:42:49.972318 10005 consensus_queue.cc:260] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.972412 10005 ts_tablet_manager.cc:1428] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:49.990074 10083 raft_consensus.cc:482] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:42:49.990191 10083 raft_consensus.cc:504] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.990356 10083 leader_election.cc:288] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601), 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211)
I20260809 00:42:49.990561 9528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8830552390b04913971e3d59527ca56c" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d77fabd64134dcab7f8e2566ef79351" is_pre_election: true
I20260809 00:42:49.990661 9528 raft_consensus.cc:2454] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 0.
I20260809 00:42:49.990706 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8830552390b04913971e3d59527ca56c" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d49785a987d455fbd83095328dc42ae" is_pre_election: true
I20260809 00:42:49.990813 9903 raft_consensus.cc:2454] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 0.
I20260809 00:42:49.990935 9623 leader_election.cc:302] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d77fabd64134dcab7f8e2566ef79351, 5d4e537a618f446cb348d1061782125d; no voters:
I20260809 00:42:49.991037 10083 raft_consensus.cc:2781] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:42:49.991065 10083 raft_consensus.cc:482] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:42:49.991088 10083 raft_consensus.cc:3037] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.991956 10083 raft_consensus.cc:504] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.992106 10083 leader_election.cc:288] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 election: Requested vote from peers 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601), 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211)
I20260809 00:42:49.992273 9528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8830552390b04913971e3d59527ca56c" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d77fabd64134dcab7f8e2566ef79351"
I20260809 00:42:49.992290 9903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8830552390b04913971e3d59527ca56c" candidate_uuid: "5d4e537a618f446cb348d1061782125d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d49785a987d455fbd83095328dc42ae"
I20260809 00:42:49.992368 9528 raft_consensus.cc:3037] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.992413 9903 raft_consensus.cc:3037] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:49.993050 9903 raft_consensus.cc:2454] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 1.
I20260809 00:42:49.993250 9627 leader_election.cc:302] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d49785a987d455fbd83095328dc42ae, 5d4e537a618f446cb348d1061782125d; no voters:
I20260809 00:42:49.993300 9528 raft_consensus.cc:2454] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d4e537a618f446cb348d1061782125d in term 1.
I20260809 00:42:49.993355 10083 raft_consensus.cc:2781] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:49.993399 10083 raft_consensus.cc:686] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 1 LEADER]: Becoming Leader. State: Replica: 5d4e537a618f446cb348d1061782125d, State: Running, Role: LEADER
I20260809 00:42:49.993455 10083 consensus_queue.cc:237] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } }
I20260809 00:42:49.994055 9314 catalog_manager.cc:5495] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d4e537a618f446cb348d1061782125d (127.30.104.194). New cstate: current_term: 1 leader_uuid: "5d4e537a618f446cb348d1061782125d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d4e537a618f446cb348d1061782125d" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 } health_report { overall_health: UNKNOWN } } }
I20260809 00:42:50.001751 10001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:49.999119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.30.104.194:41977)
0809 00:42:49.999165 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:49.999167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:49.999203 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:42:49.999281 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:49.999282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:49.999291 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:49.999338 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:42:49.999486 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:50.000015 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:50.001056 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:50.001256 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:50.001312 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:50.001482 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:42:50.001511 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:42:50.001554 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:50.001633 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:42:50.001652 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:42:50.004237 9903 raft_consensus.cc:1261] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Refusing update from remote peer 5d4e537a618f446cb348d1061782125d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:50.004231 9529 raft_consensus.cc:1261] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Refusing update from remote peer 5d4e537a618f446cb348d1061782125d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:42:50.004436 10012 consensus_queue.cc:1035] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d49785a987d455fbd83095328dc42ae" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 36211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:50.004539 10083 consensus_queue.cc:1035] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d77fabd64134dcab7f8e2566ef79351" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:42:50.017521 9314 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:50.017632 9314 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:42:50.018263 9314 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e8ef91760bc459b93e605da798a67a5: Sending DeleteTablet for 3 replicas of tablet 8830552390b04913971e3d59527ca56c
I20260809 00:42:50.018540 9501 tablet_service.cc:1514] Processing DeleteTablet for tablet 8830552390b04913971e3d59527ca56c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:50 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:51828
I20260809 00:42:50.018554 9685 tablet_service.cc:1514] Processing DeleteTablet for tablet 8830552390b04913971e3d59527ca56c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:50 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38416
I20260809 00:42:50.018666 10041 tablet_replica.cc:330] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: stopping tablet replica
I20260809 00:42:50.018668 10042 tablet_replica.cc:330] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: stopping tablet replica
I20260809 00:42:50.018721 10042 raft_consensus.cc:2229] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:50.018725 10041 raft_consensus.cc:2229] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:42:50.018766 10042 raft_consensus.cc:2258] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:50.018785 10041 raft_consensus.cc:2258] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:50.019167 10041 ts_tablet_manager.cc:1905] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:50.020877 10041 ts_tablet_manager.cc:1918] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:50.020969 10041 log.cc:1196] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/8830552390b04913971e3d59527ca56c
I20260809 00:42:50.021215 10041 ts_tablet_manager.cc:1939] T 8830552390b04913971e3d59527ca56c P 5d4e537a618f446cb348d1061782125d: Deleting consensus metadata
I20260809 00:42:50.021608 9292 catalog_manager.cc:4841] TS 5d4e537a618f446cb348d1061782125d (127.30.104.194:41977): tablet 8830552390b04913971e3d59527ca56c (table test-table [id=a7a2ed7b6b2b4c43af90802c5a588e69]) successfully deleted
I20260809 00:42:50.021768 9878 tablet_service.cc:1514] Processing DeleteTablet for tablet 8830552390b04913971e3d59527ca56c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:42:50 UTC) from {username='oryx', principal='oryx/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:35342
I20260809 00:42:50.021972 10042 ts_tablet_manager.cc:1905] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:50.023651 10042 ts_tablet_manager.cc:1918] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:50.023725 10042 log.cc:1196] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/8830552390b04913971e3d59527ca56c
I20260809 00:42:50.023988 10042 ts_tablet_manager.cc:1939] T 8830552390b04913971e3d59527ca56c P 0d77fabd64134dcab7f8e2566ef79351: Deleting consensus metadata
I20260809 00:42:50.024272 10040 tablet_replica.cc:330] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: stopping tablet replica
I20260809 00:42:50.024345 10040 raft_consensus.cc:2229] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:42:50.024400 10040 raft_consensus.cc:2258] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:42:50.024448 9292 catalog_manager.cc:4841] TS 0d77fabd64134dcab7f8e2566ef79351 (127.30.104.193:43601): tablet 8830552390b04913971e3d59527ca56c (table test-table [id=a7a2ed7b6b2b4c43af90802c5a588e69]) successfully deleted
I20260809 00:42:50.024627 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 9412
I20260809 00:42:50.027066 10040 ts_tablet_manager.cc:1905] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:42:50.028870 10040 ts_tablet_manager.cc:1918] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 00:42:50.028928 10040 log.cc:1196] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/8830552390b04913971e3d59527ca56c
I20260809 00:42:50.029177 10040 ts_tablet_manager.cc:1939] T 8830552390b04913971e3d59527ca56c P 2d49785a987d455fbd83095328dc42ae: Deleting consensus metadata
I20260809 00:42:50.033223 9297 catalog_manager.cc:4841] TS 2d49785a987d455fbd83095328dc42ae (127.30.104.195:36211): tablet 8830552390b04913971e3d59527ca56c (table test-table [id=a7a2ed7b6b2b4c43af90802c5a588e69]) successfully deleted
I20260809 00:42:50.043093 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 9600
I20260809 00:42:50.058954 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 9791
I20260809 00:42:50.071382 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 9264
2026-08-09T00:42:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2850 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:50 dist-test-slave-c12x krb5kdc[10129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:50 dist-test-slave-c12x krb5kdc[10129](info): set up 2 sockets
Aug 09 00:42:50 dist-test-slave-c12x krb5kdc[10129](info): commencing operation
krb5kdc: starting...
W20260809 00:42:51.331112 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:51 dist-test-slave-c12x krb5kdc[10129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:51.464282 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:40521
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:45755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:40521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:51.547719 10144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:51.547971 10144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:51.548010 10144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:51.549207 10144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:51.549285 10144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:51.549310 10144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:51.549338 10144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:51.549369 10144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:51.550863 10144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:45755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:40521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:40521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:51.551162 10144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:51.551403 10144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:51.556545 10149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:51.556552 10153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:51.556569 10150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:51.556763 10144 server_base.cc:1034] running on GCE node
I20260809 00:42:51.557159 10144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:51.557547 10144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:51.558758 10144 hybrid_clock.cc:648] HybridClock initialized: now 1786236171558744 us; error 49 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T00:42:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1482 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:42:51 dist-test-slave-c12x krb5kdc[10166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:42:51 dist-test-slave-c12x krb5kdc[10166](info): set up 2 sockets
Aug 09 00:42:51 dist-test-slave-c12x krb5kdc[10166](info): commencing operation
krb5kdc: starting...
W20260809 00:42:52.789235 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:42:52 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:42:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:42:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:42:52.921989 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:43115
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:43115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:53.013876 10182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:53.014132 10182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:53.014159 10182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:53.015585 10182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:53.015645 10182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:53.015661 10182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:53.015676 10182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:53.015823 10182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:53.017763 10182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:43115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:43115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:53.018052 10182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:53.018316 10182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:53.023602 10192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.023630 10190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.023638 10189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:53.023831 10182 server_base.cc:1034] running on GCE node
I20260809 00:42:53.024106 10182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:53.024439 10182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:53.025684 10182 hybrid_clock.cc:648] HybridClock initialized: now 1786236173025664 us; error 32 us; skew 500 ppm
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:53.032591 10182 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:53.033165 10182 webserver.cc:460] Webserver started at http://127.30.104.254:39161/ using document root <none> and password file <none>
I20260809 00:42:53.033463 10182 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:53.033514 10182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:53.033660 10182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:53.034853 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "07fb65f7597e4d47a7b37cba042d86f3"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "1398c76809dadb3ef3543517657de01a"
server_key_iv: "5841b2d52ef97cb281ebe7ff931e2394"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.035274 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "07fb65f7597e4d47a7b37cba042d86f3"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "1398c76809dadb3ef3543517657de01a"
server_key_iv: "5841b2d52ef97cb281ebe7ff931e2394"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.036888 10182 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:53.037830 10205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.038117 10182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:42:53.038230 10182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "07fb65f7597e4d47a7b37cba042d86f3"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "1398c76809dadb3ef3543517657de01a"
server_key_iv: "5841b2d52ef97cb281ebe7ff931e2394"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.038297 10182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:53.063192 10182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:53.065562 10182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:53.065714 10182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:53.070295 10182 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:43115
I20260809 00:42:53.070363 10283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:43115 every 8 connection(s)
I20260809 00:42:53.070796 10182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:53.071836 10284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:42:53.074309 10284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Bootstrap starting.
I20260809 00:42:53.074980 10284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Neither blocks nor log segments found. Creating new log.
I20260809 00:42:53.075312 10284 log.cc:824] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:53.076232 10284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: No bootstrap required, opened a new log
I20260809 00:42:53.077952 10284 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } }
I20260809 00:42:53.078121 10284 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:42:53.078146 10284 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fb65f7597e4d47a7b37cba042d86f3, State: Initialized, Role: FOLLOWER
I20260809 00:42:53.078322 10284 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } }
I20260809 00:42:53.078405 10284 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:53.078450 10284 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:53.078504 10284 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:42:53.078783 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 10182
I20260809 00:42:53.078876 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:42:53.079073 31139 external_mini_cluster.cc:1428] Setting key 39b2ed4223f0f114d97e1f3d4f57ca30
I20260809 00:42:53.079311 10284 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } }
I20260809 00:42:53.079473 10284 leader_election.cc:302] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07fb65f7597e4d47a7b37cba042d86f3; no voters:
I20260809 00:42:53.079881 10284 leader_election.cc:288] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:42:53.079907 10291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:42:53.080080 10291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 1 LEADER]: Becoming Leader. State: Replica: 07fb65f7597e4d47a7b37cba042d86f3, State: Running, Role: LEADER
I20260809 00:42:53.080159 10284 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:53.080191 10291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } }
I20260809 00:42:53.080551 10293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07fb65f7597e4d47a7b37cba042d86f3. Latest consensus state: current_term: 1 leader_uuid: "07fb65f7597e4d47a7b37cba042d86f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } } }
I20260809 00:42:53.080659 10293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:53.080843 10292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07fb65f7597e4d47a7b37cba042d86f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } } }
I20260809 00:42:53.080947 10292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:53.080873 10295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:53.081609 10295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:53.083876 10295 catalog_manager.cc:1347] Generated new cluster ID: 758911454edd4660893ec743dec67b68
I20260809 00:42:53.083935 10295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:53.090392 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:53.079651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49022 (local address 127.30.104.254:43115)
0809 00:42:53.079836 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:53.079847 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:53.079885 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:42:53.080027 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:53.080035 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:53.080110 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:53.081101 (+ 991us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:53.081223 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.082719 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.083683 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.084274 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.084406 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:53.086861 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:53.086865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:53.086866 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:53.086886 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:53.088669 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:53.089060 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:53.089064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:53.089064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:53.089183 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:53.089411 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:53.089412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:53.089413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:53.089689 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:53.089878 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:53.090011 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 00:42:53.090168 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260809 00:42:53.092962 10295 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:42:53.093672 10295 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:53.100389 10295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Generated new TSK 0
I20260809 00:42:53.100615 10295 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:42:53.134250 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:43115
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:53.224581 10316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:53.224813 10316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:53.224840 10316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:53.226097 10316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:53.226147 10316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:53.226177 10316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:53.227860 10316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:53.228163 10316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:53.228439 10316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:53.233901 10325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.233951 10323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.233920 10322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:53.234146 10316 server_base.cc:1034] running on GCE node
I20260809 00:42:53.234481 10316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:53.234776 10316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:53.235965 10316 hybrid_clock.cc:648] HybridClock initialized: now 1786236173235961 us; error 48 us; skew 500 ppm
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:53.238715 10316 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:42:53.239276 10316 webserver.cc:460] Webserver started at http://127.30.104.193:33713/ using document root <none> and password file <none>
I20260809 00:42:53.239609 10316 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:53.239684 10316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:53.239853 10316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:53.241091 10316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "df06173cefc247938b0bda7a2d9f4c3d"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "ec4eaf290f7bfeb5ad4582704d68dbef"
server_key_iv: "3ff021e5dac15164163c77e1a50c07c9"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.241505 10316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "df06173cefc247938b0bda7a2d9f4c3d"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "ec4eaf290f7bfeb5ad4582704d68dbef"
server_key_iv: "3ff021e5dac15164163c77e1a50c07c9"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.243115 10316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:42:53.244037 10335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.244264 10316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:53.244377 10316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "df06173cefc247938b0bda7a2d9f4c3d"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "ec4eaf290f7bfeb5ad4582704d68dbef"
server_key_iv: "3ff021e5dac15164163c77e1a50c07c9"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.244462 10316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:53.268324 10316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:53.270594 10316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:53.270784 10316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:53.271050 10316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:53.271391 10316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:53.271483 10316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.271528 10316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:53.271577 10316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.278266 10316 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:38385
I20260809 00:42:53.278314 10497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:38385 every 8 connection(s)
I20260809 00:42:53.278741 10316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:42:53.280848 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 10316
I20260809 00:42:53.281077 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:42:53.281322 31139 external_mini_cluster.cc:1428] Setting key c66485032551d49f876fa85a6742f1c5
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:53.290442 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:53.279776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:34687 (local address 127.30.104.254:43115)
0809 00:42:53.279860 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:53.279862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:53.280126 (+ 264us) server_negotiation.cc:407] Connection header received
0809 00:42:53.280865 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:53.280867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:53.280881 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:53.281062 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:53.282287 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.282858 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.284070 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.284360 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.284431 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:53.286925 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:53.286927 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:53.286928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:53.286939 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:53.288808 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:53.289294 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:53.289295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:53.289296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:53.289375 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:53.289665 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:53.289666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:53.289666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:53.289832 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:53.289908 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:53.290201 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 00:42:53.290305 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:42:53.290472 10501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:53.279740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:34687)
0809 00:42:53.279965 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:53.280006 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:53.280737 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:53.281201 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:53.281205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:53.281952 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:53.282171 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:53.282176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.282954 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.282954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:53.283978 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:53.283981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.284456 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.284457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:53.284629 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:53.285237 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:53.285250 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:53.286802 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:53.288912 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:53.288921 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:53.288923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:53.289207 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:53.289452 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:53.289453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:53.289453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:53.289591 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:53.289906 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:53.289911 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:53.290048 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 00:42:53.290146 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 00:42:53.290290 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":114,"threads_started":1}
I20260809 00:42:53.291404 10498 heartbeater.cc:347] Connected to a master server at 127.30.104.254:43115
I20260809 00:42:53.291554 10498 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:53.291730 10498 heartbeater.cc:511] Master 127.30.104.254:43115 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
I20260809 00:42:53.292337 10231 ts_manager.cc:194] Registered new tserver with Master: df06173cefc247938b0bda7a2d9f4c3d (127.30.104.193:38385)
I20260809 00:42:53.293388 10231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:34687
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:42:53.312669 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:43115
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:53.399322 10504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:53.399641 10504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:53.399684 10504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:53.400976 10504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:53.401052 10504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:53.401100 10504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:53.402693 10504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:53.403012 10504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:53.403277 10504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:53.408995 10510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.408943 10514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.408929 10512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:53.409390 10504 server_base.cc:1034] running on GCE node
I20260809 00:42:53.409636 10504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:53.409863 10504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:53.411026 10504 hybrid_clock.cc:648] HybridClock initialized: now 1786236173411014 us; error 35 us; skew 500 ppm
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:53.413825 10504 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:42:53.414374 10504 webserver.cc:460] Webserver started at http://127.30.104.194:45233/ using document root <none> and password file <none>
I20260809 00:42:53.414644 10504 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:53.414719 10504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:53.414875 10504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:53.416074 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "b99db35dcc4f4725b2f2f3ec67de5e05"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "72a159c464048e4bc35c3e5e7b612c9b"
server_key_iv: "c58493505842cc09678b1e03599b6056"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.416450 10504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "b99db35dcc4f4725b2f2f3ec67de5e05"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "72a159c464048e4bc35c3e5e7b612c9b"
server_key_iv: "c58493505842cc09678b1e03599b6056"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.417891 10504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:53.418747 10523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.418962 10504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:53.419070 10504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "b99db35dcc4f4725b2f2f3ec67de5e05"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "72a159c464048e4bc35c3e5e7b612c9b"
server_key_iv: "c58493505842cc09678b1e03599b6056"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.419148 10504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:53.433180 10504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:53.435755 10504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:53.435922 10504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:53.436190 10504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:53.436545 10504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:53.436614 10504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.436659 10504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:53.436688 10504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.443005 10504 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:36647
I20260809 00:42:53.443090 10688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:36647 every 8 connection(s)
I20260809 00:42:53.443495 10504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:53.449540 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 10504
I20260809 00:42:53.449693 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:42:53.449898 31139 external_mini_cluster.cc:1428] Setting key 588b73ee4e2ea461e9761474514b06b1
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:53.453676 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:53.444350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:40963 (local address 127.30.104.254:43115)
0809 00:42:53.444398 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:53.444400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:53.444611 (+ 211us) server_negotiation.cc:407] Connection header received
0809 00:42:53.445110 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:53.445111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:53.445120 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:53.445183 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:53.446169 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.446697 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.447877 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.448068 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.448126 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:53.450632 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:53.450634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:53.450634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:53.450642 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:53.452185 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:53.452641 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:53.452643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:53.452643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:53.452709 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:53.452976 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:53.452977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:53.452978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:53.453102 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:53.453184 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:53.453484 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 00:42:53.453580 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:42:53.453903 10691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:53.444297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:40963)
0809 00:42:53.444475 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:53.444521 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:53.445024 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:53.445239 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:53.445243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:53.445851 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:53.446063 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:53.446067 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.446787 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.446788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:53.447800 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:53.447803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.448147 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.448148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:53.448262 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:53.448868 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:53.448877 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:53.450535 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:53.452282 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:53.452290 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:53.452292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:53.452540 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:53.452788 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:53.452789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:53.452789 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:53.452908 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:53.453175 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:53.453182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:53.453341 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 00:42:53.453452 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:42:53.453637 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260809 00:42:53.454558 10689 heartbeater.cc:347] Connected to a master server at 127.30.104.254:43115
I20260809 00:42:53.454679 10689 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:53.454872 10689 heartbeater.cc:511] Master 127.30.104.254:43115 requested a full tablet report, sending...
I20260809 00:42:53.455349 10231 ts_manager.cc:194] Registered new tserver with Master: b99db35dcc4f4725b2f2f3ec67de5e05 (127.30.104.194:36647)
I20260809 00:42:53.455950 10231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:40963
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:42:53.478135 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:43115
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:53.567343 10694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:53.567680 10694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:53.567725 10694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:53.568948 10694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:53.569032 10694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:53.569082 10694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:53.570675 10694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:53.570977 10694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:53.571231 10694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:53.576798 10700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.576792 10701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.576792 10703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:53.577016 10694 server_base.cc:1034] running on GCE node
I20260809 00:42:53.577230 10694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:53.577508 10694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:53.578666 10694 hybrid_clock.cc:648] HybridClock initialized: now 1786236173578658 us; error 33 us; skew 500 ppm
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:53.581604 10694 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:42:53.582175 10694 webserver.cc:460] Webserver started at http://127.30.104.195:43121/ using document root <none> and password file <none>
I20260809 00:42:53.582501 10694 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:53.582573 10694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:53.582679 10694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:42:53.583904 10694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "8166aa0c4cd24faa8e28f3c08ef0f634"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "992df2066ffd4d18bd1a143f5c212504"
server_key_iv: "42520f16807072ecfdd5400ac283d9a5"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.584303 10694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "8166aa0c4cd24faa8e28f3c08ef0f634"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "992df2066ffd4d18bd1a143f5c212504"
server_key_iv: "42520f16807072ecfdd5400ac283d9a5"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.585853 10694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:42:53.586665 10719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.586932 10694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:53.587038 10694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "8166aa0c4cd24faa8e28f3c08ef0f634"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "992df2066ffd4d18bd1a143f5c212504"
server_key_iv: "42520f16807072ecfdd5400ac283d9a5"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.587117 10694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:53.622462 10694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:53.624993 10694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:53.625178 10694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:53.625440 10694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:53.625787 10694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:53.625844 10694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.625888 10694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:53.625917 10694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.632347 10694 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:38905
I20260809 00:42:53.632392 10891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:38905 every 8 connection(s)
I20260809 00:42:53.632738 10694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:53.635280 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 10694
I20260809 00:42:53.635432 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:42:53.635644 31139 external_mini_cluster.cc:1428] Setting key b307d82c45d7673297303e15760b0f2e
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:42:53.644006 10290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:53.633621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:55297 (local address 127.30.104.254:43115)
0809 00:42:53.633670 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:53.633672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:53.633864 (+ 192us) server_negotiation.cc:407] Connection header received
0809 00:42:53.634459 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:53.634461 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:53.634473 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:53.634545 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:53.635958 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.636759 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.637900 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.638101 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.638170 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:53.640678 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:53.640679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:53.640680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:53.640686 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:53.642172 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:53.642639 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:53.642640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:53.642641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:53.642719 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:53.643090 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:53.643091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:53.643091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:53.643263 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:53.643347 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:53.643769 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 00:42:53.643875 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:42:53.644217 10895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:53.633556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:55297)
0809 00:42:53.633724 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:53.633768 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:53.634324 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:53.634668 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:53.634673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:53.635348 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:53.635706 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:53.635713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.636846 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.636847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:53.637732 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:53.637735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:53.638210 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:53.638211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:53.638327 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:53.638970 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:53.638979 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:53.640582 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:42:53.642245 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:53.642251 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:53.642253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:53.642496 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:53.642822 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:42:53.642823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:42:53.642824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:42:53.642991 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:42:53.643350 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:42:53.643356 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:42:53.643596 (+ 240us) client_negotiation.cc:769] Sending connection context
0809 00:42:53.643737 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 00:42:53.643932 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":93,"threads_started":1}
I20260809 00:42:53.644728 10892 heartbeater.cc:347] Connected to a master server at 127.30.104.254:43115
I20260809 00:42:53.644817 10892 heartbeater.cc:464] Registering TS with master...
I20260809 00:42:53.645028 10892 heartbeater.cc:511] Master 127.30.104.254:43115 requested a full tablet report, sending...
I20260809 00:42:53.645413 10231 ts_manager.cc:194] Registered new tserver with Master: 8166aa0c4cd24faa8e28f3c08ef0f634 (127.30.104.195:38905)
I20260809 00:42:53.645789 10231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:55297
I20260809 00:42:53.647539 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab.
Entry for principal oryx/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab.
WARNING: no policy specified for oryx/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab.
Entry for principal oryx/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab.
WARNING: no policy specified for oryx/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab.
Entry for principal oryx/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab.
I20260809 00:42:53.740021 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 10316
I20260809 00:42:53.756287 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 10504
I20260809 00:42:53.767063 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 10694
I20260809 00:42:53.774492 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 10182
I20260809 00:42:53.784454 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:43115
--webserver_interface=127.30.104.254
--webserver_port=39161
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:43115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:42:53.869338 10907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:53.869580 10907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:53.869603 10907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:53.870847 10907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:42:53.870898 10907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:53.870913 10907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:53.870925 10907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:42:53.870939 10907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:42:53.872619 10907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:43115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:43115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=39161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:53.872917 10907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:53.873178 10907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:53.878494 10914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.878506 10917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:53.878494 10915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:53.878684 10907 server_base.cc:1034] running on GCE node
I20260809 00:42:53.878961 10907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:53.879281 10907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:53.880499 10907 hybrid_clock.cc:648] HybridClock initialized: now 1786236173880458 us; error 47 us; skew 500 ppm
Aug 09 00:42:53 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236173, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:53.883116 10907 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:42:53.883654 10907 webserver.cc:460] Webserver started at http://127.30.104.254:39161/ using document root <none> and password file <none>
I20260809 00:42:53.883891 10907 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:53.883950 10907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:53.885356 10907 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:53.886108 10928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:53.886334 10907 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:53.886447 10907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "07fb65f7597e4d47a7b37cba042d86f3"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "1398c76809dadb3ef3543517657de01a"
server_key_iv: "5841b2d52ef97cb281ebe7ff931e2394"
server_key_version: "encryptionkey@0"
I20260809 00:42:53.886843 10907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:53.912593 10907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:53.915150 10907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:53.915343 10907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:53.919848 10907 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:43115
I20260809 00:42:53.919912 10994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:43115 every 8 connection(s)
I20260809 00:42:53.920337 10907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:42:53.920835 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 10907
I20260809 00:42:53.921123 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:38385
--local_ip_for_outbound_sockets=127.30.104.193
--tserver_master_addrs=127.30.104.254:43115
--webserver_port=33713
--webserver_interface=127.30.104.193
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.104.193
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:42:53.922242 10995 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:42:53.923017 10995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Bootstrap starting.
I20260809 00:42:53.931797 10995 log.cc:824] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Log is configured to *not* fsync() on all Append() calls
I20260809 00:42:53.934468 10995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:42:53.934875 10995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Bootstrap complete.
I20260809 00:42:53.939083 10995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } }
I20260809 00:42:53.939551 10995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07fb65f7597e4d47a7b37cba042d86f3, State: Initialized, Role: FOLLOWER
I20260809 00:42:53.939755 10995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } }
I20260809 00:42:53.939870 10995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:42:53.939929 10995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:42:53.939982 10995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:42:53.941754 10995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } }
I20260809 00:42:53.941915 10995 leader_election.cc:302] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07fb65f7597e4d47a7b37cba042d86f3; no voters:
I20260809 00:42:53.942137 10995 leader_election.cc:288] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:42:53.942225 11002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:42:53.942411 11002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [term 2 LEADER]: Becoming Leader. State: Replica: 07fb65f7597e4d47a7b37cba042d86f3, State: Running, Role: LEADER
I20260809 00:42:53.942462 10995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:42:53.942520 11002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } }
I20260809 00:42:53.942901 11003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "07fb65f7597e4d47a7b37cba042d86f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } } }
I20260809 00:42:53.943050 11003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:53.943307 11004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07fb65f7597e4d47a7b37cba042d86f3. Latest consensus state: current_term: 2 leader_uuid: "07fb65f7597e4d47a7b37cba042d86f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07fb65f7597e4d47a7b37cba042d86f3" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43115 } } }
I20260809 00:42:53.943495 11004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:42:53.943671 11009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:42:53.944234 11009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:42:53.944388 11009 catalog_manager.cc:1259] Loaded cluster ID: 758911454edd4660893ec743dec67b68
I20260809 00:42:53.944609 11009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:42:53.945421 11009 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:42:53.945621 11009 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 07fb65f7597e4d47a7b37cba042d86f3: Loaded TSK: 0
I20260809 00:42:53.945845 11009 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:42:54.026821 10997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:54.027091 10997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:54.027134 10997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:54.028434 10997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:54.028510 10997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:54.028559 10997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:42:54.030113 10997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.193.keytab
--principal=oryx/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:38385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=33713
--tserver_master_addrs=127.30.104.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:54.030406 10997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:54.030656 10997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:54.037273 11037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:54.037303 10997 server_base.cc:1034] running on GCE node
W20260809 00:42:54.037273 11035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:54.037539 11039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:54.037760 10997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:54.038051 10997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:54.039216 10997 hybrid_clock.cc:648] HybridClock initialized: now 1786236174039205 us; error 28 us; skew 500 ppm
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236174, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:54.042089 10997 init.cc:377] Logged in from keytab as oryx/127.30.104.193@KRBTEST.COM (short username oryx)
I20260809 00:42:54.042728 10997 webserver.cc:460] Webserver started at http://127.30.104.193:33713/ using document root <none> and password file <none>
I20260809 00:42:54.043025 10997 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:54.043112 10997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:54.044808 10997 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:54.045457 11051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.045704 10997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:54.045812 10997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "df06173cefc247938b0bda7a2d9f4c3d"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "ec4eaf290f7bfeb5ad4582704d68dbef"
server_key_iv: "3ff021e5dac15164163c77e1a50c07c9"
server_key_version: "encryptionkey@0"
I20260809 00:42:54.046283 10997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:54.065543 10997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:54.068339 10997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:54.068526 10997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:54.068816 10997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:54.069211 10997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:54.069262 10997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.069294 10997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:54.069312 10997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.076900 10997 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:38385
I20260809 00:42:54.076937 11205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:38385 every 8 connection(s)
I20260809 00:42:54.077421 10997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.086586 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 10997
I20260809 00:42:54.086928 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:36647
--local_ip_for_outbound_sockets=127.30.104.194
--tserver_master_addrs=127.30.104.254:43115
--webserver_port=45233
--webserver_interface=127.30.104.194
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.104.194
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.088472 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.078377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:58969)
0809 00:42:54.078603 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.078653 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.079142 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.079770 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.079774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.080542 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.080809 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.080816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.081952 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.081953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.083243 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.083258 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.084133 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.084135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.084286 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.084932 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.084976 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.088215 (+ 3239us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
W20260809 00:42:54.088632 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.078439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:58969 (local address 127.30.104.254:43115)
0809 00:42:54.079312 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.079319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.079347 (+ 28us) server_negotiation.cc:407] Connection header received
0809 00:42:54.079524 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.079529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.079575 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.079682 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.080943 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.081845 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.083373 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.084025 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.084134 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.088436 (+ 4302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:58969: BlockingRecv error: recv got EOF from 127.30.104.193:58969 (error 108)
Metrics: {"server-negotiator.queue_time_us":824,"thread_start_us":119,"threads_started":1}
W20260809 00:42:54.089187 11206 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:43115 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.104.254:43115: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.095484 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.089447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:53713)
0809 00:42:54.089496 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.089508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.089549 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.089725 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.089726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.089903 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.089975 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.089978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.090979 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.090980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.091907 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.091911 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.092473 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.092475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.092552 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.093117 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.093122 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.095384 (+ 2262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:42:54.096063 11211 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.089513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:53713 (local address 127.30.104.254:43115)
0809 00:42:54.089552 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.089555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.089569 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:42:54.089606 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.089607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.089623 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.089702 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.090080 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.090869 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.092016 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.092375 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.092447 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.095964 (+ 3517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:53713: BlockingRecv error: recv got EOF from 127.30.104.193:53713 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.101708 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.095910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:60283)
0809 00:42:54.096267 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.096280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.096342 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.096564 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.096565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.096733 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.096776 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.096778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.097622 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.097623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.098520 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.098523 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.099065 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.099066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.099135 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.099678 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.099685 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.101630 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336}
W20260809 00:42:54.101799 11213 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.095985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:60283 (local address 127.30.104.254:43115)
0809 00:42:54.096180 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.096182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.096346 (+ 164us) server_negotiation.cc:407] Connection header received
0809 00:42:54.096413 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.096414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.096425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.096533 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.096864 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.097525 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.098622 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.098967 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.099042 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.101719 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:60283: BlockingRecv error: recv got EOF from 127.30.104.193:60283 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":107,"threads_started":1}
W20260809 00:42:54.101984 11206 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:42:54.177603 11212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:54.177881 11212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:54.177922 11212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:54.179136 11212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:54.179215 11212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:54.179263 11212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:42:54.180891 11212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.194.keytab
--principal=oryx/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:36647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=45233
--tserver_master_addrs=127.30.104.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:54.181190 11212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:54.181456 11212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:42:54.187012 11212 server_base.cc:1034] running on GCE node
W20260809 00:42:54.186993 11222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:54.186983 11221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:54.186992 11227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:54.187516 11212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:54.187808 11212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:54.189028 11212 hybrid_clock.cc:648] HybridClock initialized: now 1786236174189031 us; error 52 us; skew 500 ppm
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236174, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:54.191752 11212 init.cc:377] Logged in from keytab as oryx/127.30.104.194@KRBTEST.COM (short username oryx)
I20260809 00:42:54.192229 11212 webserver.cc:460] Webserver started at http://127.30.104.194:45233/ using document root <none> and password file <none>
I20260809 00:42:54.192492 11212 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:54.192579 11212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:54.194067 11212 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:54.194729 11238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.194919 11212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:54.195025 11212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "b99db35dcc4f4725b2f2f3ec67de5e05"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "72a159c464048e4bc35c3e5e7b612c9b"
server_key_iv: "c58493505842cc09678b1e03599b6056"
server_key_version: "encryptionkey@0"
I20260809 00:42:54.195391 11212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:54.203733 11212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:54.205914 11212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:54.206049 11212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:54.206252 11212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:54.206605 11212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:54.206643 11212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.206667 11212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:54.206715 11212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.213142 11212 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:36647
I20260809 00:42:54.213184 11383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:36647 every 8 connection(s)
I20260809 00:42:54.213562 11212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:42:54.214058 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 11212
I20260809 00:42:54.214380 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:38905
--local_ip_for_outbound_sockets=127.30.104.195
--tserver_master_addrs=127.30.104.254:43115
--webserver_port=43121
--webserver_interface=127.30.104.195
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.104.195
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 00:42:54.221956 11213 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.214448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:37837 (local address 127.30.104.254:43115)
0809 00:42:54.214534 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.214537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.214882 (+ 345us) server_negotiation.cc:407] Connection header received
0809 00:42:54.215390 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.215391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.215408 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.215509 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.216475 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.217280 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.218417 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.218631 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.218715 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.221897 (+ 3182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:37837: BlockingRecv error: recv got EOF from 127.30.104.194:37837 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:42:54.222015 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.214390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:37837)
0809 00:42:54.214749 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.214798 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.215296 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.215583 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.215587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.216160 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.216359 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.216372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.217421 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.217422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.218273 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.218275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.218722 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.218723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.218876 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.219486 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.219498 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.221796 (+ 2298us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":104,"threads_started":1}
W20260809 00:42:54.222291 11384 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:43115 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.104.254:43115: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.227798 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.222520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:47517)
0809 00:42:54.222609 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.222623 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.222694 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.222845 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.222846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.223032 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.223095 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.223097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.223860 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.223861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.224776 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.224780 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.225172 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.225174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.225248 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.225695 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.225701 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.227720 (+ 2019us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 00:42:54.227942 11213 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.222589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:47517 (local address 127.30.104.254:43115)
0809 00:42:54.222630 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.222632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.222678 (+ 46us) server_negotiation.cc:407] Connection header received
0809 00:42:54.222761 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.222762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.222772 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.222817 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.223180 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.223779 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.224895 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.225086 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.225150 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.227829 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:47517: BlockingRecv error: recv got EOF from 127.30.104.194:47517 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.232971 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.228296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:34549)
0809 00:42:54.228349 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.228361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.228404 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.228541 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.228542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.228654 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.228682 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.228683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.229608 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.229608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.230089 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.230091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.230567 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.230568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.230614 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.231096 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.231100 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.232919 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 00:42:54.233127 11213 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.228377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:34549 (local address 127.30.104.254:43115)
0809 00:42:54.228417 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.228419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.228426 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:42:54.228456 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.228457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.228467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.228532 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.228748 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.229529 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.230165 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.230470 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.230537 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.233052 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:34549: BlockingRecv error: recv got EOF from 127.30.104.194:34549 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 00:42:54.233140 11384 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:42:54.307930 11390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:42:54.308159 11390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:42:54.308185 11390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:42:54.309448 11390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:42:54.309504 11390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:42:54.309532 11390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:42:54.311108 11390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/krb5kdc/oryx_127.30.104.195.keytab
--principal=oryx/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:38905
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=43121
--tserver_master_addrs=127.30.104.254:43115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:42:54.311386 11390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:42:54.311712 11390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:42:54.317211 11399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:54.317229 11390 server_base.cc:1034] running on GCE node
W20260809 00:42:54.317194 11398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:42:54.317194 11402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:42:54.317672 11390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:42:54.317945 11390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:42:54.319118 11390 hybrid_clock.cc:648] HybridClock initialized: now 1786236174319103 us; error 30 us; skew 500 ppm
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236174, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:42:54.321906 11390 init.cc:377] Logged in from keytab as oryx/127.30.104.195@KRBTEST.COM (short username oryx)
I20260809 00:42:54.322391 11390 webserver.cc:460] Webserver started at http://127.30.104.195:43121/ using document root <none> and password file <none>
I20260809 00:42:54.322660 11390 fs_manager.cc:359] Metadata directory not provided
I20260809 00:42:54.322719 11390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:42:54.324290 11390 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:42:54.325006 11411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.325249 11390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:42:54.325348 11390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "8166aa0c4cd24faa8e28f3c08ef0f634"
format_stamp: "Formatted at 2026-08-09 00:42:53 on dist-test-slave-c12x"
server_key: "992df2066ffd4d18bd1a143f5c212504"
server_key_iv: "42520f16807072ecfdd5400ac283d9a5"
server_key_version: "encryptionkey@0"
I20260809 00:42:54.325754 11390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:42:54.337951 11390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:42:54.340472 11390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:42:54.340633 11390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:42:54.340896 11390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:42:54.341231 11390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:42:54.341264 11390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.341308 11390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:42:54.341334 11390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:42:54.348058 11390 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:38905
I20260809 00:42:54.348129 11572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:38905 every 8 connection(s)
I20260809 00:42:54.348486 11390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:42:54.351099 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 11390
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.357632 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.349297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:38733)
0809 00:42:54.349500 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.349538 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.350106 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.350326 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.350329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.350925 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.351193 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.351199 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.352214 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.352215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.353346 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.353349 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.353871 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.353872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.353998 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.354636 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.354646 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.357451 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
W20260809 00:42:54.357649 11213 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.349322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:38733 (local address 127.30.104.254:43115)
0809 00:42:54.349384 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.349386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.349631 (+ 245us) server_negotiation.cc:407] Connection header received
0809 00:42:54.350205 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.350206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.350216 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.350270 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.351317 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.352126 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.353469 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.353778 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.353858 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.357572 (+ 3714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:38733: BlockingRecv error: recv got EOF from 127.30.104.195:38733 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 00:42:54.357900 11573 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:43115 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.104.254:43115: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
I20260809 00:42:54.360085 11211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.351729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49030 (local address 127.30.104.254:43115)
0809 00:42:54.351782 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.351784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.351884 (+ 100us) server_negotiation.cc:407] Connection header received
0809 00:42:54.351968 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.351968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.351979 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.352017 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.352683 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.353395 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.354264 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.354594 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.354791 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.356137 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:42:54.356142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:42:54.356143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:42:54.356156 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:42:54.358368 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:54.358883 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:54.358887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:54.358888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:54.359042 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:42:54.359261 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:42:54.359262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:42:54.359262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:42:54.359590 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:42:54.359739 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 00:42:54.359870 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 00:42:54.359969 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.363225 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.358147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:36699)
0809 00:42:54.358202 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.358213 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.358259 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.358392 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.358393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.358570 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.358633 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.358636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.359717 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.359718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.360519 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.360522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.360934 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.360935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.361012 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.361380 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.361385 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.363132 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 00:42:54.363279 11213 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.358205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:36699 (local address 127.30.104.254:43115)
0809 00:42:54.358245 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.358247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.358256 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:54.358306 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.358306 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.358316 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.358382 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.358872 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.359640 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.360608 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.360852 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.360936 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.363214 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:36699: BlockingRecv error: recv got EOF from 127.30.104.195:36699 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:54 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:54.368387 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:54.363662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:36367)
0809 00:42:54.363716 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:54.363727 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:54.363767 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:54.363907 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:54.363908 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:54.364076 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:54.364117 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.364120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.365015 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.365016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.365633 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:54.365635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.366013 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.366014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:54.366069 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.366526 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:54.366531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:54.368294 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:42:54.368438 11213 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:54.363745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:36367 (local address 127.30.104.254:43115)
0809 00:42:54.363793 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:54.363796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:54.363804 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:42:54.363829 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:54.363829 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:54.363838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:54.363887 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:54.364190 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.364932 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.365719 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:54.365929 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:54.366014 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:54.368371 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:36367: BlockingRecv error: recv got EOF from 127.30.104.195:36367 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 00:42:54.368607 11573 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 00:42:55 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:55 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:55.107889 11578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:55.102463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:42887)
0809 00:42:55.102664 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:55.102675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:55.102739 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:55.102933 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:55.102933 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:55.103120 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:55.103171 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:55.103173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.104049 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.104050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:55.104653 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:55.104655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.105080 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.105082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:55.105132 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:55.105690 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:55.105698 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:55.107812 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":101,"threads_started":1}
W20260809 00:42:55.108081 11580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:55.102614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:42887 (local address 127.30.104.254:43115)
0809 00:42:55.102784 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:55.102786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:55.102795 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:55.102824 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:55.102825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:55.102835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:55.102907 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:55.103252 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.103945 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.104780 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.104995 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.105062 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:55.107958 (+ 2896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:42887: BlockingRecv error: recv got EOF from 127.30.104.193:42887 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":77,"threads_started":1}
Aug 09 00:42:55 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:55 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:55.238579 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:55.233561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:43981)
0809 00:42:55.233708 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:55.233720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:55.233787 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:55.233993 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:55.233995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:55.234161 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:55.234222 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:55.234224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.234912 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.234913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:55.235556 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:55.235559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.235986 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.235987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:55.236036 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:55.236663 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:55.236669 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:55.238506 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
W20260809 00:42:55.238814 11580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:55.233642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:43981 (local address 127.30.104.254:43115)
0809 00:42:55.233709 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:55.233711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:55.233798 (+ 87us) server_negotiation.cc:407] Connection header received
0809 00:42:55.233846 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:55.233846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:55.233880 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:55.233926 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:55.234302 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.234809 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.235661 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.235881 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.235955 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:55.238652 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:43981: BlockingRecv error: recv got EOF from 127.30.104.194:43981 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 00:42:55 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:55 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:55.374586 11583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:55.369254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:44755)
0809 00:42:55.369410 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:55.369442 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:55.369512 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:55.369740 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:55.369741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:55.369928 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:55.369976 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:55.369978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.370664 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.370665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:55.371235 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:55.371237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.371687 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.371688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:55.371753 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:55.372245 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:55.372250 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:55.374468 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
W20260809 00:42:55.374631 11580 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:55.369398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:44755 (local address 127.30.104.254:43115)
0809 00:42:55.369475 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:55.369477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:55.369491 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:42:55.369615 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:55.369616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:55.369631 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:55.369720 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:55.370061 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.370576 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.371325 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:55.371600 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:55.371675 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:55.374561 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:44755: BlockingRecv error: recv got EOF from 127.30.104.195:44755 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 00:42:56 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:56 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:56.114629 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:56.108545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:54253)
0809 00:42:56.108815 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:56.108828 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:56.108906 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:56.109063 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:56.109064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:56.109254 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:56.109308 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:56.109310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.110117 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.110118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:56.110801 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:56.110805 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.111502 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.111503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:56.111582 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:56.112203 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:56.112212 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:56.114563 (+ 2351us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":133,"threads_started":1}
W20260809 00:42:56.115072 11586 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:56.108693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:54253 (local address 127.30.104.254:43115)
0809 00:42:56.108846 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:56.108848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:56.108906 (+ 58us) server_negotiation.cc:407] Connection header received
0809 00:42:56.108961 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:56.108962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:56.108973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:56.109036 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:56.109377 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.109992 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.110909 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.111249 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.111309 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:56.115016 (+ 3707us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:54253: BlockingRecv error: recv got EOF from 127.30.104.193:54253 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
Aug 09 00:42:56 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:56 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:56.244518 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:56.239265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:47345)
0809 00:42:56.239544 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:56.239558 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:56.239638 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:56.239791 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:56.239792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:56.239984 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:56.240036 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:56.240038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.240795 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.240796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:56.241326 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:56.241328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.241774 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.241775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:56.241907 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:56.242394 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:56.242399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:56.244405 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":242,"threads_started":1}
W20260809 00:42:56.244683 11586 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:56.239397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:47345 (local address 127.30.104.254:43115)
0809 00:42:56.239502 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:56.239504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:56.239655 (+ 151us) server_negotiation.cc:407] Connection header received
0809 00:42:56.239690 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:56.239692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:56.239705 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:56.239765 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:56.240130 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.240691 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.241442 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.241670 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.241739 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:56.244595 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:47345: BlockingRecv error: recv got EOF from 127.30.104.194:47345 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 00:42:56 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:56 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:56.380852 11588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:56.375231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:38037)
0809 00:42:56.375452 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:56.375465 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:56.375529 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:56.375780 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:56.375781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:56.375960 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:56.376011 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:56.376013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.376761 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.376762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:56.377289 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:56.377291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.377713 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.377714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:56.377762 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:56.378408 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:56.378413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:56.380792 (+ 2379us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":91,"threads_started":1}
W20260809 00:42:56.381067 11586 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:56.375511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:38037 (local address 127.30.104.254:43115)
0809 00:42:56.375597 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:56.375599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:56.375610 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:56.375649 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:56.375650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:56.375663 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:56.375717 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:56.376124 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.376655 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.377429 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:56.377630 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:56.377689 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:56.380983 (+ 3294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:38037: BlockingRecv error: recv got EOF from 127.30.104.195:38037 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 00:42:57 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:57 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:57.120874 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:57.115539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:57987)
0809 00:42:57.115686 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:57.115700 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:57.115769 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:57.115991 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:57.115993 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:57.116182 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:57.116236 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:57.116237 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.116971 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.116972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:57.117783 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:57.117786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.118284 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.118286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:57.118356 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:57.118975 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:57.118981 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:57.120814 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
W20260809 00:42:57.121104 11593 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:57.115656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:57987 (local address 127.30.104.254:43115)
0809 00:42:57.115798 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:57.115800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:57.115809 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:57.115838 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:57.115839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:57.115849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:57.115927 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:57.116331 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.116872 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.117909 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.118203 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.118269 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:57.121021 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:57987: BlockingRecv error: recv got EOF from 127.30.104.193:57987 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
Aug 09 00:42:57 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:57 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:57.250660 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:57.245075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:42677)
0809 00:42:57.245282 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:57.245296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:57.245377 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:57.245589 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:57.245590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:57.245793 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:57.245845 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:57.245847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.246560 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.246561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:57.247163 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:57.247165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.247622 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.247623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:57.247687 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:57.248225 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:57.248230 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:57.250596 (+ 2366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":106,"threads_started":1}
W20260809 00:42:57.250818 11593 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:57.245187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:42677 (local address 127.30.104.254:43115)
0809 00:42:57.245272 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:57.245274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:57.245366 (+ 92us) server_negotiation.cc:407] Connection header received
0809 00:42:57.245447 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:57.245448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:57.245461 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:57.245541 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:57.245934 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.246461 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.247265 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.247545 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.247622 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:57.250724 (+ 3102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:42677: BlockingRecv error: recv got EOF from 127.30.104.194:42677 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 00:42:57 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:57 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:57.386656 11596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:57.381578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:35431)
0809 00:42:57.381753 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:57.381771 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:57.381829 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:57.381981 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:57.381982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:57.382167 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:57.382218 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:57.382220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.382873 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.382874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:57.383411 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:57.383412 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.383880 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.383881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:57.383926 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:57.384513 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:57.384518 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:57.386594 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":122,"threads_started":1}
W20260809 00:42:57.386840 11593 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:57.381660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:35431 (local address 127.30.104.254:43115)
0809 00:42:57.381754 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:57.381755 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:57.381817 (+ 62us) server_negotiation.cc:407] Connection header received
0809 00:42:57.381910 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:57.381910 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:57.381925 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:57.381970 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:57.382287 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.382792 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.383566 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:57.383792 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:57.383880 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:57.386752 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:35431: BlockingRecv error: recv got EOF from 127.30.104.195:35431 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 00:42:58 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:58 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:58.130332 11597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:58.121393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:50085)
0809 00:42:58.121558 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:58.121570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:58.121631 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:58.121826 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:58.121827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:58.122003 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:58.122057 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:58.122059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.122763 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.122764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:58.123318 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:58.123320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.123778 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.123779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:58.123830 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:58.128226 (+ 4396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:58.128231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:58.130271 (+ 2040us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
W20260809 00:42:58.130528 11598 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:58.121501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:50085 (local address 127.30.104.254:43115)
0809 00:42:58.121640 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:58.121642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:58.121652 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:42:58.121677 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:58.121678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:58.121689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:58.121754 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:58.122143 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.122671 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.123409 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.123686 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.123749 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:58.130440 (+ 6691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:50085: BlockingRecv error: recv got EOF from 127.30.104.193:50085 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
Aug 09 00:42:58 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:58 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:58.256616 11599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:58.251237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:56255)
0809 00:42:58.251429 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:58.251473 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:58.251581 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:58.251761 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:58.251762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:58.251954 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:58.252006 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:58.252008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.252703 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.252705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:58.253258 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:58.253259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.253632 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.253633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:58.253715 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:58.254286 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:58.254292 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:58.256548 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":103,"threads_started":1}
W20260809 00:42:58.256795 11598 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:58.251341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:56255 (local address 127.30.104.254:43115)
0809 00:42:58.251417 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:58.251420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:58.251556 (+ 136us) server_negotiation.cc:407] Connection header received
0809 00:42:58.251656 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:58.251661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:58.251674 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:58.251738 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:58.252091 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.252605 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.253348 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.253549 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.253633 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:58.256701 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:56255: BlockingRecv error: recv got EOF from 127.30.104.194:56255 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 00:42:58 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:58 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:58.392527 11600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:58.387359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:42613)
0809 00:42:58.387565 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:58.387578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:58.387672 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:58.387834 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:58.387835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:58.388113 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:58.388182 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:58.388185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.388910 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.388911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:58.389527 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:58.389529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.389917 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.389918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:58.389988 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:58.390523 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:58.390528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:58.392425 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":133,"threads_started":1}
W20260809 00:42:58.392585 11598 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:58.387502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:42613 (local address 127.30.104.254:43115)
0809 00:42:58.387558 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:58.387560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:58.387646 (+ 86us) server_negotiation.cc:407] Connection header received
0809 00:42:58.387731 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:58.387732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:58.387748 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:58.387827 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:58.388278 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.388809 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.389644 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:58.389840 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:58.389919 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:58.392520 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:42613: BlockingRecv error: recv got EOF from 127.30.104.195:42613 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 00:42:59 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:59 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:59.135887 11601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:59.130963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:52013)
0809 00:42:59.131129 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:59.131140 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:59.131191 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:59.131336 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:59.131337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:59.131566 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:59.131642 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:59.131645 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.132373 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.132373 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:59.132912 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:59.132914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.133276 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.133277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:59.133329 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:59.133881 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:59.133886 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:59.135819 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
W20260809 00:42:59.136066 11602 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:59.131043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:52013 (local address 127.30.104.254:43115)
0809 00:42:59.131206 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:59.131208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:59.131217 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:42:59.131241 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:59.131242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:59.131253 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:59.131323 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:59.131755 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.132289 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.133007 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.133204 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.133262 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:59.135944 (+ 2682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:52013: BlockingRecv error: recv got EOF from 127.30.104.193:52013 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
Aug 09 00:42:59 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:59 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:59.262251 11603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:59.257206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:46567)
0809 00:42:59.257350 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:59.257363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:59.257448 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:59.257606 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:59.257607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:59.257814 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:59.257880 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:59.257882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.258569 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.258570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:59.259114 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:59.259115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.259538 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.259539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:59.259617 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:59.260197 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:59.260202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:59.262188 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
W20260809 00:42:59.262430 11602 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:59.257358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:46567 (local address 127.30.104.254:43115)
0809 00:42:59.257468 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:59.257470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:59.257481 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:42:59.257513 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:59.257514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:59.257533 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:59.257601 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:59.257969 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.258482 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.259208 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.259412 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.259530 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:59.262345 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:46567: BlockingRecv error: recv got EOF from 127.30.104.194:46567 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 00:42:59 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:42:59 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:42:59.397585 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:42:59.392977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:47517)
0809 00:42:59.393159 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 00:42:59.393170 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:42:59.393238 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:42:59.393367 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:42:59.393367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:42:59.393532 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:42:59.393582 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:59.393584 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.394207 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.394208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:59.394731 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:42:59.394732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.395042 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.395043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:42:59.395090 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:59.395660 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:42:59.395665 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:42:59.397484 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
W20260809 00:42:59.397637 11602 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:42:59.393063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:47517 (local address 127.30.104.254:43115)
0809 00:42:59.393166 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 00:42:59.393168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:42:59.393238 (+ 70us) server_negotiation.cc:407] Connection header received
0809 00:42:59.393285 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:42:59.393285 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:42:59.393298 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:42:59.393346 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:42:59.393648 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.394141 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.394805 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:42:59.394979 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:42:59.395031 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:42:59.397572 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:47517: BlockingRecv error: recv got EOF from 127.30.104.195:47517 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 00:43:00 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:00 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:00.141491 11606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:00.136382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:58253)
0809 00:43:00.136662 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:00.136676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:00.136745 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:00.136886 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:00.136887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:00.137076 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:00.137132 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:00.137134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.137822 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.137823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:00.138387 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:00.138389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.138790 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.138791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:00.138848 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:00.139319 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:00.139324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:00.141378 (+ 2054us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":74,"threads_started":1}
W20260809 00:43:00.141554 11607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:00.136448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:58253 (local address 127.30.104.254:43115)
0809 00:43:00.136741 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:00.136743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:00.136752 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:43:00.136795 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:00.136796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:00.136806 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:00.136867 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:00.137202 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.137741 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.138468 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.138663 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.138718 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:00.141474 (+ 2756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:58253: BlockingRecv error: recv got EOF from 127.30.104.193:58253 (error 108)
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":78,"threads_started":1}
Aug 09 00:43:00 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:00 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:00.267908 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:00.262802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:52933)
0809 00:43:00.262976 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:00.262986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:00.263047 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:00.263189 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:00.263189 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:00.263363 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:00.263414 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:00.263415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.264151 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.264152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:00.264713 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:00.264715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.265048 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.265049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:00.265090 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:00.265713 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:00.265720 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:00.267847 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":95,"threads_started":1}
W20260809 00:43:00.268107 11607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:00.262924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:52933 (local address 127.30.104.254:43115)
0809 00:43:00.262994 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:00.262996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:00.263042 (+ 46us) server_negotiation.cc:407] Connection header received
0809 00:43:00.263094 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:00.263095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:00.263109 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:00.263169 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:00.263551 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.264068 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.264785 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.264979 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.265034 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:00.268031 (+ 2997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:52933: BlockingRecv error: recv got EOF from 127.30.104.194:52933 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 00:43:00 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:00 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:00.403505 11609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:00.398286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:39129)
0809 00:43:00.398534 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:00.398562 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:00.398654 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:00.398815 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:00.398816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:00.399056 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:00.399135 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:00.399137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.399899 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.399900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:00.400439 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:00.400466 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.400833 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.400834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:00.400899 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:00.401531 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:00.401536 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:00.403375 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":75,"threads_started":1}
W20260809 00:43:00.403565 11607 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:00.398379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:39129 (local address 127.30.104.254:43115)
0809 00:43:00.398434 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:00.398436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:00.398652 (+ 216us) server_negotiation.cc:407] Connection header received
0809 00:43:00.398723 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:00.398728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:00.398743 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:00.398785 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:00.399221 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.399811 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.400553 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:00.400734 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:00.400827 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:00.403499 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:39129: BlockingRecv error: recv got EOF from 127.30.104.195:39129 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 00:43:01 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:01 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:01.147980 11610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:01.142110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:43139)
0809 00:43:01.142398 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:01.142413 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:01.142547 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:01.142722 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:01.142723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:01.142902 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:01.142993 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:01.142995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.144041 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.144042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:01.144609 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:01.144611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.144983 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.144984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:01.145033 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:01.145728 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:01.145734 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:01.147901 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":105,"threads_started":1}
W20260809 00:43:01.148149 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:01.142179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:43139 (local address 127.30.104.254:43115)
0809 00:43:01.142520 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:01.142522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:01.142533 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:43:01.142624 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:01.142625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:01.142637 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:01.142700 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:01.143130 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.143949 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.144699 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.144912 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.144968 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:01.148044 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:43139: BlockingRecv error: recv got EOF from 127.30.104.193:43139 (error 108)
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":201,"threads_started":1}
Aug 09 00:43:01 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:01 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:01.273317 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:01.268525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:57523)
0809 00:43:01.268715 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:01.268727 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:01.268819 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:01.269010 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:01.269011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:01.269196 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:01.269246 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:01.269248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.269934 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.269935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:01.270487 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:01.270489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.270854 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.270855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:01.270904 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:01.271401 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:01.271405 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:01.273246 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":82,"threads_started":1}
W20260809 00:43:01.273490 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:01.268643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:57523 (local address 127.30.104.254:43115)
0809 00:43:01.268701 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:01.268703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:01.268880 (+ 177us) server_negotiation.cc:407] Connection header received
0809 00:43:01.268918 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:01.268919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:01.268932 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:01.268989 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:01.269315 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.269825 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.270580 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.270776 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.270836 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:01.273366 (+ 2530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:57523: BlockingRecv error: recv got EOF from 127.30.104.194:57523 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 00:43:01 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:01 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:01.409247 11614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:01.404084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:57683)
0809 00:43:01.404232 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:01.404241 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:01.404288 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:01.404554 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:01.404556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:01.404761 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:01.404813 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:01.404814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.405540 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.405542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:01.406078 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:01.406080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.406476 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.406477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:01.406551 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:01.407141 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:01.407147 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:01.409139 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":58,"threads_started":1}
W20260809 00:43:01.409301 11611 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:01.404292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:57683 (local address 127.30.104.254:43115)
0809 00:43:01.404404 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:01.404409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:01.404418 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:43:01.404452 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:01.404453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:01.404468 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:01.404543 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:01.404909 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.405444 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.406175 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:01.406388 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:01.406477 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:01.409235 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:57683: BlockingRecv error: recv got EOF from 127.30.104.195:57683 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 00:43:02 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:02 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:02.153424 11615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:02.148570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:58339)
0809 00:43:02.148725 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:02.148736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:02.148788 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:02.149000 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:02.149001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:02.149197 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:02.149251 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:02.149253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.149959 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.149960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:02.150537 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:02.150539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.150872 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.150873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:02.150919 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:02.151419 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:02.151424 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:02.153355 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
W20260809 00:43:02.153563 11616 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:02.148727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:58339 (local address 127.30.104.254:43115)
0809 00:43:02.148867 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:02.148869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:02.148876 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:43:02.148898 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:02.148898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:02.148907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:02.148987 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:02.149351 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.149881 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.150613 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.150806 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.150858 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:02.153467 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:58339: BlockingRecv error: recv got EOF from 127.30.104.193:58339 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
Aug 09 00:43:02 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:02 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:02.279175 11620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:02.273934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:36011)
0809 00:43:02.274153 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:02.274167 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:02.274264 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:02.274436 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:02.274437 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:02.274643 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:02.274693 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:02.274695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.275577 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.275578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:02.276117 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:02.276119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.276505 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.276506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:02.276566 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:02.277086 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:02.277092 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:02.279113 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":117,"threads_started":1}
W20260809 00:43:02.279286 11616 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:02.274049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:36011 (local address 127.30.104.254:43115)
0809 00:43:02.274155 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:02.274157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:02.274243 (+ 86us) server_negotiation.cc:407] Connection header received
0809 00:43:02.274335 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:02.274336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:02.274349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:02.274427 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:02.274777 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.275500 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.276210 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.276424 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.276505 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:02.279215 (+ 2710us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:36011: BlockingRecv error: recv got EOF from 127.30.104.194:36011 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 00:43:02 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:02 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:02.415122 11621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:02.409906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:60143)
0809 00:43:02.410087 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:02.410102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:02.410187 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:02.410348 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:02.410349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:02.410541 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:02.410594 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:02.410596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.411284 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.411285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:02.411949 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:02.411952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.412309 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.412310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:02.412353 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:02.412834 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:02.412840 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:02.415058 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
W20260809 00:43:02.415290 11616 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:02.410053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:60143 (local address 127.30.104.254:43115)
0809 00:43:02.410128 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:02.410130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:02.410167 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:43:02.410250 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:02.410251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:02.410264 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:02.410315 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:02.410663 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.411180 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.412047 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:02.412238 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:02.412294 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:02.415185 (+ 2891us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:60143: BlockingRecv error: recv got EOF from 127.30.104.195:60143 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 00:43:03 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:03 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:03.163887 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:03.153952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:40851)
0809 00:43:03.154130 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:03.154142 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:03.154198 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:03.154346 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:03.154346 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:03.154521 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:03.154572 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:03.154574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.155311 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.155312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:03.155955 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:03.155958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.156318 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.156319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:03.156362 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:03.161690 (+ 5328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:03.161695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:03.163815 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
W20260809 00:43:03.164026 11626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:03.154076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:40851 (local address 127.30.104.254:43115)
0809 00:43:03.154225 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:03.154226 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:03.154234 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:43:03.154258 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:03.154258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:03.154268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:03.154329 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:03.154713 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.155233 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.156047 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.156248 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.156306 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:03.163925 (+ 7619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:40851: BlockingRecv error: recv got EOF from 127.30.104.193:40851 (error 108)
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
Aug 09 00:43:03 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:03 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:03.285498 11627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:03.279847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:48935)
0809 00:43:03.279986 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:03.280004 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:03.280097 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:03.280240 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:03.280241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:03.280524 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:03.280586 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:03.280589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.281293 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.281295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:03.281857 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:03.281859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.282264 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.282265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:03.282324 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:03.282834 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:03.282839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:03.285424 (+ 2585us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
W20260809 00:43:03.285632 11626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:03.279924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:48935 (local address 127.30.104.254:43115)
0809 00:43:03.279998 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:03.280000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:03.280088 (+ 88us) server_negotiation.cc:407] Connection header received
0809 00:43:03.280162 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:03.280163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:03.280179 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:03.280232 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:03.280658 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.281195 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.281955 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.282172 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.282263 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:03.285531 (+ 3268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:48935: BlockingRecv error: recv got EOF from 127.30.104.194:48935 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 00:43:03 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:03 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:03.420993 11630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:03.415755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:57405)
0809 00:43:03.415969 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:03.415984 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:03.416090 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:03.416310 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:03.416311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:03.416513 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:03.416558 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:03.416559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.417255 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.417257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:03.417787 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:03.417789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.418269 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.418270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:03.418341 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:03.418885 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:03.418890 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:03.420921 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
W20260809 00:43:03.421137 11626 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:03.415899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:57405 (local address 127.30.104.254:43115)
0809 00:43:03.415992 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:03.415994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:03.416070 (+ 76us) server_negotiation.cc:407] Connection header received
0809 00:43:03.416190 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:03.416192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:03.416206 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:03.416303 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:03.416646 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.417163 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.417905 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:03.418207 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:03.418269 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:03.421047 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:57405: BlockingRecv error: recv got EOF from 127.30.104.195:57405 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 00:43:04 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:04 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:04.170125 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:04.164502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:39093)
0809 00:43:04.164775 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:04.164791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:04.164872 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:04.165032 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:04.165032 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:04.165221 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:04.165273 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:04.165274 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.166007 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.166009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:04.166564 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:04.166566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.166945 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.166946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:04.167003 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:04.167634 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:04.167639 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:04.170060 (+ 2421us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":81,"threads_started":1}
W20260809 00:43:04.170274 11632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:04.164704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:39093 (local address 127.30.104.254:43115)
0809 00:43:04.164875 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:04.164877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:04.164913 (+ 36us) server_negotiation.cc:407] Connection header received
0809 00:43:04.164940 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:04.164940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:04.164951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:04.165006 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:04.165383 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.165899 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.166654 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.166851 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.166910 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:04.170173 (+ 3263us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:39093: BlockingRecv error: recv got EOF from 127.30.104.193:39093 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
Aug 09 00:43:04 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:04 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:04.290975 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:04.286123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:44055)
0809 00:43:04.286282 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:04.286294 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:04.286368 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:04.286520 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:04.286521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:04.286702 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:04.286756 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:04.286757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.287567 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.287571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:04.288095 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:04.288096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.288432 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.288433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:04.288476 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:04.288942 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:04.288946 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:04.290911 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
W20260809 00:43:04.291138 11632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:04.286232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:44055 (local address 127.30.104.254:43115)
0809 00:43:04.286292 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:04.286294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:04.286350 (+ 56us) server_negotiation.cc:407] Connection header received
0809 00:43:04.286419 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:04.286419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:04.286430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:04.286500 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:04.286856 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.287403 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.288167 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.288358 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.288414 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:04.291059 (+ 2645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:44055: BlockingRecv error: recv got EOF from 127.30.104.194:44055 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 00:43:04 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:04 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:04.426900 11634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:04.421604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:57953)
0809 00:43:04.421806 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:04.421823 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:04.421901 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:04.422044 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:04.422045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:04.422234 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:04.422288 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:04.422289 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.422926 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.422927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:04.423571 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:04.423573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.423905 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.423906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:04.423954 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:04.424564 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:04.424569 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:04.426837 (+ 2268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":136,"threads_started":1}
W20260809 00:43:04.427026 11632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:04.421757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:57953 (local address 127.30.104.254:43115)
0809 00:43:04.421816 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:04.421818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:04.421883 (+ 65us) server_negotiation.cc:407] Connection header received
0809 00:43:04.421952 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:04.421952 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:04.421964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:04.422017 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:04.422356 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.422847 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.423648 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:04.423830 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:04.423877 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:04.426945 (+ 3068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:57953: BlockingRecv error: recv got EOF from 127.30.104.195:57953 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 00:43:05 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:05 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:05.176118 11635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:05.170698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:41185)
0809 00:43:05.170890 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:05.170903 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:05.170971 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:05.171194 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:05.171195 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:05.171508 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:05.171572 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:05.171575 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.172317 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.172318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:05.172959 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:05.172961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.173354 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.173355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:05.173408 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:05.173994 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:05.173999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:05.176046 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":121,"threads_started":1}
W20260809 00:43:05.176277 11636 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:05.170840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:41185 (local address 127.30.104.254:43115)
0809 00:43:05.171030 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:05.171032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:05.171042 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:43:05.171067 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:05.171068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:05.171080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:05.171162 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:05.171706 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.172224 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.173052 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.173262 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.173321 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:05.176182 (+ 2861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:41185: BlockingRecv error: recv got EOF from 127.30.104.193:41185 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":126,"threads_started":1}
Aug 09 00:43:05 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:05 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:05.296730 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:05.291665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:53809)
0809 00:43:05.291797 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:05.291810 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:05.291867 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:05.292018 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:05.292018 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:05.292195 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:05.292241 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:05.292243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.293025 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.293026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:05.293577 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:05.293579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.293991 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.293992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:05.294034 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:05.294592 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:05.294597 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:05.296655 (+ 2058us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
W20260809 00:43:05.297006 11636 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:05.291769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:53809 (local address 127.30.104.254:43115)
0809 00:43:05.291847 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:05.291853 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:05.291862 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:43:05.291932 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:05.291932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:05.291953 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:05.292010 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:05.292364 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.292926 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.293680 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.293917 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.293982 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:05.296850 (+ 2868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:53809: BlockingRecv error: recv got EOF from 127.30.104.194:53809 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 00:43:05 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:05 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:05.432755 11639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:05.427530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:42483)
0809 00:43:05.427740 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:05.427754 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:05.427854 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:05.428031 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:05.428032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:05.428303 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:05.428361 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:05.428367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.429089 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.429090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:05.429730 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:05.429732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.430153 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.430153 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:05.430215 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:05.430716 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:05.430721 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:05.432639 (+ 1918us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":101,"threads_started":1}
W20260809 00:43:05.432806 11636 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:05.427625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:42483 (local address 127.30.104.254:43115)
0809 00:43:05.427724 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:05.427727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:05.427832 (+ 105us) server_negotiation.cc:407] Connection header received
0809 00:43:05.427924 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:05.427925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:05.427938 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:05.428020 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:05.428449 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.428988 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.429854 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:05.430061 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:05.430150 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:05.432734 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:42483: BlockingRecv error: recv got EOF from 127.30.104.195:42483 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 00:43:06 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:06 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:06.181623 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:06.176652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:51095)
0809 00:43:06.176810 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:06.176821 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:06.176871 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:06.177018 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:06.177018 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:06.177214 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:06.177261 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:06.177262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.177996 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.177997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:06.178552 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:06.178554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.178910 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.178911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:06.178955 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:06.179520 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:06.179527 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:06.181509 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
W20260809 00:43:06.181669 11641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:06.176720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:51095 (local address 127.30.104.254:43115)
0809 00:43:06.176895 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:06.176897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:06.176907 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:43:06.176931 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:06.176931 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:06.176941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:06.177013 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:06.177360 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.177909 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.178642 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.178841 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.178900 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:06.181593 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:51095: BlockingRecv error: recv got EOF from 127.30.104.193:51095 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
Aug 09 00:43:06 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:06 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:06.302731 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:06.297235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:51185)
0809 00:43:06.297440 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:06.297456 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:06.297536 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:06.297733 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:06.297734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:06.297958 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:06.298009 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:06.298011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.298835 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.298836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:06.299562 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:06.299565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.300027 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.300029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:06.300097 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:06.300609 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:06.300614 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:06.302612 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
W20260809 00:43:06.302767 11641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:06.297349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:51185 (local address 127.30.104.254:43115)
0809 00:43:06.297445 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:06.297448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:06.297511 (+ 63us) server_negotiation.cc:407] Connection header received
0809 00:43:06.297609 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:06.297610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:06.297625 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:06.297722 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:06.298114 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.298736 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.299669 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.299902 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.300021 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:06.302698 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:51185: BlockingRecv error: recv got EOF from 127.30.104.194:51185 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 00:43:06 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:06 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:06.438547 11645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:06.433342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:52029)
0809 00:43:06.433689 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:06.433702 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:06.433803 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:06.433970 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:06.433971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:06.434187 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:06.434236 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:06.434238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.434940 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.434941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:06.435519 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:06.435521 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.435901 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.435902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:06.435963 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:06.436495 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:06.436500 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:06.438442 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":153,"threads_started":1}
W20260809 00:43:06.438607 11641 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:06.433475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:52029 (local address 127.30.104.254:43115)
0809 00:43:06.433679 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:06.433681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:06.433783 (+ 102us) server_negotiation.cc:407] Connection header received
0809 00:43:06.433865 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:06.433866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:06.433882 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:06.433962 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:06.434329 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.434848 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.435615 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:06.435812 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:06.435898 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:06.438538 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:52029: BlockingRecv error: recv got EOF from 127.30.104.195:52029 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Aug 09 00:43:07 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:07 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:07.187342 11646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:07.182453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:44367)
0809 00:43:07.182633 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:07.182644 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:07.182703 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:07.182849 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:07.182849 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:07.183026 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:07.183082 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:07.183083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.183750 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.183751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:07.184278 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:07.184280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.184716 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.184717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:07.184769 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:07.185298 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:07.185303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:07.187282 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
W20260809 00:43:07.187579 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:07.182487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:44367 (local address 127.30.104.254:43115)
0809 00:43:07.182629 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:07.182632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:07.182708 (+ 76us) server_negotiation.cc:407] Connection header received
0809 00:43:07.182754 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:07.182755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:07.182766 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:07.182829 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:07.183151 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.183681 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.184379 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.184605 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.184672 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:07.187501 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:44367: BlockingRecv error: recv got EOF from 127.30.104.193:44367 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
Aug 09 00:43:07 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:07 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:07.308660 11648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:07.303439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:43371)
0809 00:43:07.303681 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:07.303692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:07.303755 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:07.303917 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:07.303918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:07.304117 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:07.304169 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:07.304171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.305017 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.305018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:07.305587 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:07.305589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.305978 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.305980 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:07.306041 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:07.306511 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:07.306516 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:07.308555 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":178,"threads_started":1}
W20260809 00:43:07.308717 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:07.303609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:43371 (local address 127.30.104.254:43115)
0809 00:43:07.303691 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:07.303693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:07.303758 (+ 65us) server_negotiation.cc:407] Connection header received
0809 00:43:07.303815 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:07.303816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:07.303828 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:07.303893 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:07.304242 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.304925 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.305686 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.305895 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.305976 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:07.308650 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:43371: BlockingRecv error: recv got EOF from 127.30.104.194:43371 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 00:43:07 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:07 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:07.444913 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:07.439214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:33253)
0809 00:43:07.439446 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:07.439487 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:07.439600 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:07.439799 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:07.439801 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:07.440031 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:07.440080 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:07.440082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.440840 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.440841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:07.441448 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:07.441450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.441847 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.441848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:07.441922 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:07.442481 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:07.442487 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:07.444802 (+ 2315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":115,"threads_started":1}
W20260809 00:43:07.444963 11647 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:07.439332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:33253 (local address 127.30.104.254:43115)
0809 00:43:07.439469 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:07.439471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:07.439575 (+ 104us) server_negotiation.cc:407] Connection header received
0809 00:43:07.439686 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:07.439687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:07.439705 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:07.439790 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:07.440191 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.440746 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.441548 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:07.441761 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:07.441847 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:07.444892 (+ 3045us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:33253: BlockingRecv error: recv got EOF from 127.30.104.195:33253 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 00:43:08 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:08 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:08.196466 11652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:08.187950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:58033)
0809 00:43:08.188126 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:08.188142 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:08.188207 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:08.188342 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:08.188343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:08.188527 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:08.188580 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:08.188581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.189273 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.189275 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:08.189860 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:08.189862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.190202 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.190203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:08.190251 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:08.194377 (+ 4126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:08.194413 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:08.196405 (+ 1992us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
W20260809 00:43:08.196668 11654 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:08.188097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:58033 (local address 127.30.104.254:43115)
0809 00:43:08.188217 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:08.188219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:08.188228 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:43:08.188257 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:08.188257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:08.188266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:08.188326 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:08.188652 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.189169 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.189935 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.190128 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.190201 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:08.196564 (+ 6363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:58033: BlockingRecv error: recv got EOF from 127.30.104.193:58033 (error 108)
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":55,"threads_started":1}
Aug 09 00:43:08 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:08 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:08.314733 11655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:08.309275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:56107)
0809 00:43:08.309509 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:08.309523 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:08.309630 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:08.309824 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:08.309825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:08.310028 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:08.310077 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:08.310079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.310822 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.310824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:08.311381 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:08.311383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.311960 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.311961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:08.312054 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:08.312647 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:08.312652 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:08.314669 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":128,"threads_started":1}
W20260809 00:43:08.314883 11654 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:08.309423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:56107 (local address 127.30.104.254:43115)
0809 00:43:08.309518 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:08.309521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:08.309608 (+ 87us) server_negotiation.cc:407] Connection header received
0809 00:43:08.309724 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:08.309725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:08.309744 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:08.309814 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:08.310168 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.310721 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.311549 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.311837 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.311959 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:08.314797 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:56107: BlockingRecv error: recv got EOF from 127.30.104.194:56107 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 00:43:08 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:08 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.195@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:08.451160 11658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:08.445581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.195:57611)
0809 00:43:08.445795 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:08.445809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:08.445896 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:08.446077 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:08.446079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:08.446362 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:08.446424 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:08.446426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.447114 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.447114 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:08.447828 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:08.447830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.448200 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.448201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:08.448251 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:08.448794 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:08.448799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:08.451013 (+ 2214us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":106,"threads_started":1}
W20260809 00:43:08.451238 11654 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:08.445669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:57611 (local address 127.30.104.254:43115)
0809 00:43:08.445744 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:08.445746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:08.445910 (+ 164us) server_negotiation.cc:407] Connection header received
0809 00:43:08.445950 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:08.445951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:08.445969 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:08.446045 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:08.446511 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.447035 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.447924 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:08.448120 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:08.448181 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:08.451159 (+ 2978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.195:57611: BlockingRecv error: recv got EOF from 127.30.104.195:57611 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 00:43:09 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:09 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.193@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:09.202508 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:09.197079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.193:56769)
0809 00:43:09.197232 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:09.197250 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:09.197302 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:09.197578 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:09.197579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:09.197805 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:09.197861 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:09.197863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.198525 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.198526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:09.199148 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:09.199150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.199518 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.199519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:09.199572 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.200083 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:09.200088 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:09.202393 (+ 2305us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
W20260809 00:43:09.202562 11660 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:09.197259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:56769 (local address 127.30.104.254:43115)
0809 00:43:09.197422 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:09.197424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:09.197434 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:43:09.197470 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:09.197471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:09.197482 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:09.197573 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:09.197939 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.198450 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.199226 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.199421 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.199501 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.202485 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.193:56769: BlockingRecv error: recv got EOF from 127.30.104.193:56769 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
Aug 09 00:43:09 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 00:43:09 dist-test-slave-c12x krb5kdc[10166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.104.194@KRBTEST.COM for oryx/127.30.104.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 00:43:09.320286 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:09.315379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:43115 (local address 127.30.104.194:59039)
0809 00:43:09.315583 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:09.315598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:09.315678 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:09.315826 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:09.315826 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:09.316024 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:09.316079 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:09.316080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.316770 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.316771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:09.317350 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:09.317352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.317699 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.317699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:09.317743 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.318301 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:09.318305 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:09.320211 (+ 1906us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.104.254:43115: Server oryx/127.30.104.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":118,"threads_started":1}
W20260809 00:43:09.320365 11660 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:09.315553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:59039 (local address 127.30.104.254:43115)
0809 00:43:09.315641 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:09.315643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:09.315656 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:43:09.315737 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:09.315738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:09.315749 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:09.315800 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:09.316152 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.316677 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.317431 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.317630 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.317681 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.320314 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.104.194:59039: BlockingRecv error: recv got EOF from 127.30.104.194:59039 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 00:43:09.355379 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 10997
I20260809 00:43:09.367043 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 11212
I20260809 00:43:09.378160 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 11390
I20260809 00:43:09.393024 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 10907
2026-08-09T00:43:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17837 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T00:43:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:43:09Z Disabled control of system clock
I20260809 00:43:09.430868 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44067
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:44067
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 00:43:09.517087 11669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:09.517324 11669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:09.517345 11669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:09.518671 11669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:43:09.518718 11669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:09.518730 11669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:09.518743 11669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:43:09.518755 11669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:43:09.520242 11669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:44067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:09.520478 11669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:09.520749 11669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:09.526403 11676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:09.526528 11669 server_base.cc:1034] running on GCE node
W20260809 00:43:09.526465 11679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:09.526458 11677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:09.527005 11669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:09.527318 11669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:09.528488 11669 hybrid_clock.cc:648] HybridClock initialized: now 1786236189528486 us; error 70 us; skew 500 ppm
I20260809 00:43:09.529821 11669 webserver.cc:460] Webserver started at http://127.30.104.254:45525/ using document root <none> and password file <none>
I20260809 00:43:09.530119 11669 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:09.530172 11669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:09.530357 11669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:09.531567 11669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "ec37051ac0344ad68b20e648d7162f93"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "876b70f65a9043c1ff43371b9b43749f"
server_key_iv: "90c0422ddf4c9b4d39277f60f08a77f4"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.531965 11669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "ec37051ac0344ad68b20e648d7162f93"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "876b70f65a9043c1ff43371b9b43749f"
server_key_iv: "90c0422ddf4c9b4d39277f60f08a77f4"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.533346 11669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:43:09.534242 11688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:09.534554 11669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:43:09.534641 11669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "ec37051ac0344ad68b20e648d7162f93"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "876b70f65a9043c1ff43371b9b43749f"
server_key_iv: "90c0422ddf4c9b4d39277f60f08a77f4"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.534744 11669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:09.572535 11669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:09.572916 11669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:09.573079 11669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:09.577512 11669 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:44067
I20260809 00:43:09.577580 11759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:44067 every 8 connection(s)
I20260809 00:43:09.577977 11669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:43:09.578804 11760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:43:09.581655 11760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93: Bootstrap starting.
I20260809 00:43:09.582297 11760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93: Neither blocks nor log segments found. Creating new log.
I20260809 00:43:09.582634 11760 log.cc:824] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93: Log is configured to *not* fsync() on all Append() calls
I20260809 00:43:09.583549 11760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93: No bootstrap required, opened a new log
I20260809 00:43:09.585192 11760 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec37051ac0344ad68b20e648d7162f93" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44067 } }
I20260809 00:43:09.585371 11760 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:43:09.585419 11760 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec37051ac0344ad68b20e648d7162f93, State: Initialized, Role: FOLLOWER
I20260809 00:43:09.585628 11760 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec37051ac0344ad68b20e648d7162f93" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44067 } }
I20260809 00:43:09.585736 11760 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:43:09.585780 11760 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:43:09.585830 11760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:43:09.586403 11760 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec37051ac0344ad68b20e648d7162f93" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44067 } }
I20260809 00:43:09.586537 11760 leader_election.cc:302] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec37051ac0344ad68b20e648d7162f93; no voters:
I20260809 00:43:09.586792 11760 leader_election.cc:288] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:43:09.586829 11767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:43:09.586942 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 11669
I20260809 00:43:09.586978 11767 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [term 1 LEADER]: Becoming Leader. State: Replica: ec37051ac0344ad68b20e648d7162f93, State: Running, Role: LEADER
I20260809 00:43:09.587046 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:43:09.587071 11767 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec37051ac0344ad68b20e648d7162f93" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44067 } }
I20260809 00:43:09.587106 11760 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:43:09.587248 31139 external_mini_cluster.cc:1428] Setting key ad415adc70ba69ebd5691d31b1695eb5
I20260809 00:43:09.587387 11767 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec37051ac0344ad68b20e648d7162f93. Latest consensus state: current_term: 1 leader_uuid: "ec37051ac0344ad68b20e648d7162f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec37051ac0344ad68b20e648d7162f93" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44067 } } }
I20260809 00:43:09.587594 11767 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [sys.catalog]: This master's current role is: LEADER
I20260809 00:43:09.587630 11768 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec37051ac0344ad68b20e648d7162f93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec37051ac0344ad68b20e648d7162f93" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44067 } } }
I20260809 00:43:09.587805 11768 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93 [sys.catalog]: This master's current role is: LEADER
I20260809 00:43:09.587889 11780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:43:09.588456 11780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:43:09.589960 11795 mvcc.cc:205] Tried to move back new op lower bound from 7316423432558854144 to 7316423432549015552. Current Snapshot: MvccSnapshot[applied={T|T < 7316423432558854144}]
I20260809 00:43:09.590425 11780 catalog_manager.cc:1347] Generated new cluster ID: d1bd1ae9479d40f8a14e8621cd48e990
I20260809 00:43:09.590487 11780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:43:09.593849 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:09.587996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39936 (local address 127.30.104.254:44067)
0809 00:43:09.588244 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:09.588255 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:09.588328 (+ 73us) server_negotiation.cc:407] Connection header received
0809 00:43:09.588532 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:09.588541 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:09.588623 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:09.588842 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:09.589087 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.590504 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.591633 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.592261 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.592396 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.593104 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:09.593108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:09.593108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:43:09.593328 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:09.593371 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:43:09.593387 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:09.593476 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:09.593540 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 00:43:09.593626 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":124,"threads_started":1}
I20260809 00:43:09.625643 11780 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:43:09.626180 11780 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:43:09.632632 11780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec37051ac0344ad68b20e648d7162f93: Generated new TSK 0
I20260809 00:43:09.632858 11780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:43:09.644004 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44067
--builtin_ntp_servers=127.30.104.212:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 00:43:09.730006 11799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:09.730285 11799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:09.730309 11799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:09.731559 11799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:09.731602 11799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:09.731627 11799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:43:09.733031 11799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:09.733304 11799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:09.733623 11799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:09.739210 11808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:09.739199 11804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:09.739279 11799 server_base.cc:1034] running on GCE node
W20260809 00:43:09.739199 11806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:09.739884 11799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:09.740125 11799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:09.741314 11799 hybrid_clock.cc:648] HybridClock initialized: now 1786236189741302 us; error 32 us; skew 500 ppm
I20260809 00:43:09.742800 11799 webserver.cc:460] Webserver started at http://127.30.104.193:45359/ using document root <none> and password file <none>
I20260809 00:43:09.743062 11799 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:09.743142 11799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:09.743304 11799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:09.744534 11799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "bbc0233884f34402a185c59da6235cb7"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "bd44327489a299e7c9bfbe0382032c54"
server_key_iv: "1dabf752a35a74a97398c532a0196677"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.744937 11799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "bbc0233884f34402a185c59da6235cb7"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "bd44327489a299e7c9bfbe0382032c54"
server_key_iv: "1dabf752a35a74a97398c532a0196677"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.746392 11799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:43:09.747272 11817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:09.747550 11799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:43:09.747664 11799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "bbc0233884f34402a185c59da6235cb7"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "bd44327489a299e7c9bfbe0382032c54"
server_key_iv: "1dabf752a35a74a97398c532a0196677"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.747758 11799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:09.770996 11799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:09.771344 11799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:09.771535 11799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:09.771814 11799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:09.772186 11799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:09.772245 11799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:09.772291 11799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:09.772316 11799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:09.778112 11799 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:44073
I20260809 00:43:09.778208 11983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:44073 every 8 connection(s)
I20260809 00:43:09.778576 11799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:43:09.780565 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 11799
I20260809 00:43:09.780675 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:43:09.780853 31139 external_mini_cluster.cc:1428] Setting key 976e185ea388b3cde3959429a829067e
I20260809 00:43:09.782135 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44067
--builtin_ntp_servers=127.30.104.212:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 00:43:09.785259 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:09.779585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:60827 (local address 127.30.104.254:44067)
0809 00:43:09.779652 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:09.779654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:09.779919 (+ 265us) server_negotiation.cc:407] Connection header received
0809 00:43:09.780488 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:09.780490 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:09.780513 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:09.780604 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:09.781650 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.782405 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.783470 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.783804 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.783880 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.784917 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:09.784919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:09.784919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:43:09.784926 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:09.784946 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:43:09.784952 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:09.785014 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:09.785147 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 00:43:09.785187 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:43:09.785543 11987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:09.779535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44067 (local address 127.30.104.193:60827)
0809 00:43:09.779756 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:09.779801 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:09.780338 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:09.780720 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:09.780725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:09.780736 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:09.781529 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:09.781536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.782516 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.782517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:09.783347 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:09.783350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.783966 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.783968 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:09.784136 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.784757 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:43:09.784757 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:09.784790 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:43:09.784816 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:09.785030 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:09.785037 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:09.785041 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:43:09.785134 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:43:09.785264 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":98,"threads_started":1}
I20260809 00:43:09.786049 11984 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44067
I20260809 00:43:09.786162 11984 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:09.786357 11984 heartbeater.cc:511] Master 127.30.104.254:44067 requested a full tablet report, sending...
I20260809 00:43:09.786978 11708 ts_manager.cc:194] Registered new tserver with Master: bbc0233884f34402a185c59da6235cb7 (127.30.104.193:44073)
I20260809 00:43:09.787779 11708 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.193:60827
W20260809 00:43:09.870292 11988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:09.870561 11988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:09.870610 11988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:09.871892 11988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:09.871968 11988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:09.872011 11988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:43:09.873452 11988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:09.873778 11988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:09.874126 11988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:09.879374 12001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:09.879385 11998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:09.879380 11997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:09.879662 11988 server_base.cc:1034] running on GCE node
I20260809 00:43:09.879911 11988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:09.880198 11988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:09.881372 11988 hybrid_clock.cc:648] HybridClock initialized: now 1786236189881370 us; error 43 us; skew 500 ppm
I20260809 00:43:09.882825 11988 webserver.cc:460] Webserver started at http://127.30.104.194:38597/ using document root <none> and password file <none>
I20260809 00:43:09.883105 11988 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:09.883180 11988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:09.883335 11988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:09.884624 11988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "5226e1579e6047ba9332f768a733943f"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "723756694d5c607dc76b51b3d5578f57"
server_key_iv: "e2bbf09d0dfd4fc77fa9392834c17e5f"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.885021 11988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "5226e1579e6047ba9332f768a733943f"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "723756694d5c607dc76b51b3d5578f57"
server_key_iv: "e2bbf09d0dfd4fc77fa9392834c17e5f"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.886375 11988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:43:09.887207 12010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:09.887511 11988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:43:09.887611 11988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "5226e1579e6047ba9332f768a733943f"
format_stamp: "Formatted at 2026-08-09 00:43:09 on dist-test-slave-c12x"
server_key: "723756694d5c607dc76b51b3d5578f57"
server_key_iv: "e2bbf09d0dfd4fc77fa9392834c17e5f"
server_key_version: "encryptionkey@0"
I20260809 00:43:09.887699 11988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:09.900822 11988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:09.901167 11988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:09.901331 11988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:09.901592 11988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:09.901953 11988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:09.902009 11988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:09.902050 11988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:09.902076 11988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:09.907963 11988 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:35955
I20260809 00:43:09.908016 12168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:35955 every 8 connection(s)
I20260809 00:43:09.908361 11988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:43:09.909250 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 11988
I20260809 00:43:09.909363 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:43:09.909565 31139 external_mini_cluster.cc:1428] Setting key 581d7c4367764a57ed417b99ff7da57d
I20260809 00:43:09.910641 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44067
--builtin_ntp_servers=127.30.104.212:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 00:43:09.915079 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:09.909164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:33005 (local address 127.30.104.254:44067)
0809 00:43:09.909231 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:09.909233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:09.909574 (+ 341us) server_negotiation.cc:407] Connection header received
0809 00:43:09.910305 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:09.910322 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:09.910336 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:09.910409 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:09.911189 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.911973 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.913489 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.913683 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.913758 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.914682 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:09.914683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:09.914684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:43:09.914690 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:09.914710 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:43:09.914715 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:09.914824 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:09.914919 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 00:43:09.914955 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:43:09.915218 12172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:09.909166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44067 (local address 127.30.104.194:33005)
0809 00:43:09.909420 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:09.909468 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:09.910191 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:09.910481 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:09.910485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:09.910494 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:09.911090 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:09.911096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.912086 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.912086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:09.913394 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:09.913397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:09.913778 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:09.913779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:09.913924 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:09.914528 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:43:09.914529 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:09.914560 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:43:09.914585 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:09.914837 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:09.914842 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:09.914845 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:43:09.914902 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:43:09.914994 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":105,"threads_started":1}
I20260809 00:43:09.915804 12169 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44067
I20260809 00:43:09.915927 12169 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:09.916167 12169 heartbeater.cc:511] Master 127.30.104.254:44067 requested a full tablet report, sending...
I20260809 00:43:09.916639 11708 ts_manager.cc:194] Registered new tserver with Master: 5226e1579e6047ba9332f768a733943f (127.30.104.194:35955)
I20260809 00:43:09.917213 11708 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.194:33005
W20260809 00:43:09.997617 12173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:09.997820 12173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:09.997843 12173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:09.998962 12173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:09.999001 12173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:09.999027 12173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:43:10.000347 12173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:10.000558 12173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:10.000830 12173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:10.006255 12179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:10.006270 12181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:10.006270 12183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:10.006527 12173 server_base.cc:1034] running on GCE node
I20260809 00:43:10.006852 12173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:10.007128 12173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:10.008301 12173 hybrid_clock.cc:648] HybridClock initialized: now 1786236190008277 us; error 44 us; skew 500 ppm
I20260809 00:43:10.009847 12173 webserver.cc:460] Webserver started at http://127.30.104.195:42575/ using document root <none> and password file <none>
I20260809 00:43:10.010141 12173 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:10.010215 12173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:10.010375 12173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:10.011616 12173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "7b4e869cde1c42ce8a12cfaaebbe20a3"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "0e94df6947bfe77e09b0ef020068e2e5"
server_key_iv: "717b1638e25f47ee93e3c5d6bc3fe1ce"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.011996 12173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "7b4e869cde1c42ce8a12cfaaebbe20a3"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "0e94df6947bfe77e09b0ef020068e2e5"
server_key_iv: "717b1638e25f47ee93e3c5d6bc3fe1ce"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.013409 12173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:43:10.014318 12193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.014521 12173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:43:10.014632 12173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "7b4e869cde1c42ce8a12cfaaebbe20a3"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "0e94df6947bfe77e09b0ef020068e2e5"
server_key_iv: "717b1638e25f47ee93e3c5d6bc3fe1ce"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.014824 12173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:10.036551 12173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:10.036917 12173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:10.037089 12173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:10.037375 12173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:10.037840 12173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:10.037914 12173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.037947 12173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:10.037966 12173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.044023 12173 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:42449
I20260809 00:43:10.044122 12347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:42449 every 8 connection(s)
I20260809 00:43:10.044485 12173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:43:10.048228 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 12173
I20260809 00:43:10.048333 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:43:10.048528 31139 external_mini_cluster.cc:1428] Setting key 24bef5436d95cd54239ac5282a42c8cf
I20260809 00:43:10.050866 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.045357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:44487 (local address 127.30.104.254:44067)
0809 00:43:10.045454 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:10.045456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:10.045692 (+ 236us) server_negotiation.cc:407] Connection header received
0809 00:43:10.046211 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:10.046213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:10.046225 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:10.046300 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:10.047062 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:10.047692 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:10.049241 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:10.049581 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:10.049662 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:10.050601 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:10.050602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:10.050602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:43:10.050607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:10.050623 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:43:10.050627 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:10.050694 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:10.050773 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:43:10.050803 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:43:10.051122 12350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.045297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44067 (local address 127.30.104.195:44487)
0809 00:43:10.045511 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:10.045555 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:10.046093 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:10.046384 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:10.046387 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:10.046397 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:10.046934 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:10.046938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:10.047780 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:10.047781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:10.049082 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:10.049086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:10.049682 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:10.049683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:10.049818 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:10.050444 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:43:10.050444 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:10.050475 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:43:10.050501 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:10.050699 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:10.050704 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:10.050707 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:43:10.050762 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 00:43:10.050854 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":104,"threads_started":1}
I20260809 00:43:10.051826 12348 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44067
I20260809 00:43:10.051944 12348 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:10.052138 12348 heartbeater.cc:511] Master 127.30.104.254:44067 requested a full tablet report, sending...
I20260809 00:43:10.052503 11708 ts_manager.cc:194] Registered new tserver with Master: 7b4e869cde1c42ce8a12cfaaebbe20a3 (127.30.104.195:42449)
I20260809 00:43:10.052897 11708 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.195:44487
I20260809 00:43:10.060755 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:43:10.062968 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 11799
W20260809 00:43:10.062923 11784 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:10.062278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.30.104.254:44067)
0809 00:43:10.062359 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:10.062361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:10.062545 (+ 184us) server_negotiation.cc:407] Connection header received
0809 00:43:10.062576 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:10.062576 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:10.062590 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:10.062644 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:10.062828 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39950: BlockingRecv error: recv got EOF from 127.0.0.1:39950 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:43:10.073822 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 11988
I20260809 00:43:10.087016 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 12173
I20260809 00:43:10.096143 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 11669
2026-08-09T00:43:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (701 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T00:43:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:43:10Z Disabled control of system clock
I20260809 00:43:10.132054 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:36027
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:37077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:36027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:43:10.216823 12364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:10.217046 12364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:10.217069 12364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:10.218273 12364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:43:10.218322 12364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:10.218336 12364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:10.218348 12364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:43:10.218369 12364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:43:10.219869 12364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:36027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:36027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:10.220145 12364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:10.220486 12364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:10.225827 12372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:10.225852 12375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:10.225971 12364 server_base.cc:1034] running on GCE node
W20260809 00:43:10.225896 12373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:10.226308 12364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:10.226630 12364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:10.227787 12364 hybrid_clock.cc:648] HybridClock initialized: now 1786236190227775 us; error 54 us; skew 500 ppm
I20260809 00:43:10.229123 12364 webserver.cc:460] Webserver started at http://127.30.104.254:39543/ using document root <none> and password file <none>
I20260809 00:43:10.229396 12364 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:10.229449 12364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:10.229628 12364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:10.230786 12364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "453652f425a340a89b3e728e28b14642"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "20accca62d16a8afc1abfac15facbf48"
server_key_iv: "c3d7348ebbf20b54e4898a69cc3c8c8a"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.231179 12364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "453652f425a340a89b3e728e28b14642"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "20accca62d16a8afc1abfac15facbf48"
server_key_iv: "c3d7348ebbf20b54e4898a69cc3c8c8a"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.232751 12364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:43:10.233654 12383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.233867 12364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:43:10.233984 12364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "453652f425a340a89b3e728e28b14642"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "20accca62d16a8afc1abfac15facbf48"
server_key_iv: "c3d7348ebbf20b54e4898a69cc3c8c8a"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.234077 12364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:10.237720 12364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:10.238023 12364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:10.238173 12364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:10.242333 12364 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:36027
I20260809 00:43:10.242369 12454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:36027 every 8 connection(s)
I20260809 00:43:10.242754 12364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:43:10.243592 12456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:43:10.246399 12456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642: Bootstrap starting.
I20260809 00:43:10.246963 12456 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642: Neither blocks nor log segments found. Creating new log.
I20260809 00:43:10.247282 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 12364
I20260809 00:43:10.247391 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:43:10.247371 12456 log.cc:824] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642: Log is configured to *not* fsync() on all Append() calls
I20260809 00:43:10.247681 31139 external_mini_cluster.cc:1428] Setting key 0a86e68c073c8285eb81d0eb75869562
I20260809 00:43:10.248270 12456 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642: No bootstrap required, opened a new log
I20260809 00:43:10.250312 12456 raft_consensus.cc:348] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453652f425a340a89b3e728e28b14642" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36027 } }
I20260809 00:43:10.250468 12456 raft_consensus.cc:374] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:43:10.250496 12456 raft_consensus.cc:729] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 453652f425a340a89b3e728e28b14642, State: Initialized, Role: FOLLOWER
I20260809 00:43:10.250636 12456 consensus_queue.cc:260] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453652f425a340a89b3e728e28b14642" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36027 } }
I20260809 00:43:10.250723 12456 raft_consensus.cc:388] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:43:10.250762 12456 raft_consensus.cc:482] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:43:10.250833 12456 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:43:10.250986 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.248275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54520 (local address 127.30.104.254:36027)
0809 00:43:10.248532 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:10.248539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:10.248564 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:43:10.249438 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:10.249446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:10.249485 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:10.249678 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:10.250390 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:10.250393 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:10.250394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:43:10.250398 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:10.250453 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:43:10.250479 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:10.250595 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:10.250710 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:43:10.250774 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260809 00:43:10.252045 12456 raft_consensus.cc:504] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453652f425a340a89b3e728e28b14642" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36027 } }
I20260809 00:43:10.252184 12456 leader_election.cc:302] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 453652f425a340a89b3e728e28b14642; no voters:
I20260809 00:43:10.252377 12456 leader_election.cc:288] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:43:10.252421 12462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:43:10.252610 12462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [term 1 LEADER]: Becoming Leader. State: Replica: 453652f425a340a89b3e728e28b14642, State: Running, Role: LEADER
I20260809 00:43:10.252696 12456 sys_catalog.cc:564] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:43:10.252722 12462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453652f425a340a89b3e728e28b14642" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36027 } }
I20260809 00:43:10.253065 12462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 453652f425a340a89b3e728e28b14642. Latest consensus state: current_term: 1 leader_uuid: "453652f425a340a89b3e728e28b14642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453652f425a340a89b3e728e28b14642" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36027 } } }
I20260809 00:43:10.253077 12464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "453652f425a340a89b3e728e28b14642" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "453652f425a340a89b3e728e28b14642" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36027 } } }
I20260809 00:43:10.253244 12462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [sys.catalog]: This master's current role is: LEADER
I20260809 00:43:10.253355 12464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642 [sys.catalog]: This master's current role is: LEADER
I20260809 00:43:10.253845 12473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:43:10.254496 12473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:43:10.256163 12473 catalog_manager.cc:1347] Generated new cluster ID: 8a7e45723c434c759ae0231e74f3e431
I20260809 00:43:10.256219 12473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:43:10.259835 12473 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:43:10.260030 12473 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:43:10.264268 12473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 453652f425a340a89b3e728e28b14642: Generated new TSK 0
I20260809 00:43:10.264475 12473 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:43:10.273128 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36027
--builtin_ntp_servers=127.30.104.212:37077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:43:10.361016 12484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:10.361261 12484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:10.361287 12484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:10.362797 12484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:10.362869 12484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:10.362915 12484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:43:10.365361 12484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12484
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:10.365700 12484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:10.366074 12484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:10.371714 12496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:10.371731 12493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:10.371712 12492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:10.372303 12484 server_base.cc:1034] running on GCE node
I20260809 00:43:10.372535 12484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:10.372802 12484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:10.373993 12484 hybrid_clock.cc:648] HybridClock initialized: now 1786236190373980 us; error 32 us; skew 500 ppm
I20260809 00:43:10.375615 12484 webserver.cc:460] Webserver started at http://127.30.104.193:40103/ using document root <none> and password file <none>
I20260809 00:43:10.375917 12484 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:10.375978 12484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:10.376116 12484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:10.377671 12484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "02263c5a0cb84466917d171c388741aa"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "b5439b10a2b4888a46f3dfa48854996d"
server_key_iv: "5bd91dd3150d938ffddc50c50e66552e"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.378130 12484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "02263c5a0cb84466917d171c388741aa"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "b5439b10a2b4888a46f3dfa48854996d"
server_key_iv: "5bd91dd3150d938ffddc50c50e66552e"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.379807 12484 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260809 00:43:10.380729 12506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.381048 12484 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:43:10.381131 12484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "02263c5a0cb84466917d171c388741aa"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "b5439b10a2b4888a46f3dfa48854996d"
server_key_iv: "5bd91dd3150d938ffddc50c50e66552e"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.381204 12484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:10.386196 12484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:10.386510 12484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:10.386641 12484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:10.386865 12484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:10.387221 12484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:10.387267 12484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.387295 12484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:10.387311 12484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.393791 12484 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:44469
I20260809 00:43:10.393941 12665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:44469 every 8 connection(s)
I20260809 00:43:10.394248 12484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:43:10.397655 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.395105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:35095 (local address 127.30.104.254:36027)
0809 00:43:10.395181 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:10.395183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:10.395467 (+ 284us) server_negotiation.cc:407] Connection header received
0809 00:43:10.396046 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:10.396048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:10.396054 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:10.396125 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:10.397423 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:10.397424 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:10.397424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:43:10.397429 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:10.397444 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:43:10.397448 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:10.397505 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:10.397585 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:43:10.397598 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:43:10.397919 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.395070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36027 (local address 127.30.104.193:35095)
0809 00:43:10.395302 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:10.395345 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:10.395898 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:10.396208 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:10.396211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:10.396220 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:10.397250 (+ 1030us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:43:10.397250 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:10.397282 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:43:10.397315 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:10.397515 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:10.397517 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:10.397520 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:43:10.397576 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:43:10.397661 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
I20260809 00:43:10.398402 12666 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36027
I20260809 00:43:10.398514 12666 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:10.398679 12666 heartbeater.cc:511] Master 127.30.104.254:36027 requested a full tablet report, sending...
I20260809 00:43:10.399294 12404 ts_manager.cc:194] Registered new tserver with Master: 02263c5a0cb84466917d171c388741aa (127.30.104.193:44469)
I20260809 00:43:10.399379 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 12484
I20260809 00:43:10.399504 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:43:10.399725 31139 external_mini_cluster.cc:1428] Setting key 9f69b13a889ea2a06cd9f58ea27eb347
I20260809 00:43:10.400688 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36027
--builtin_ntp_servers=127.30.104.212:37077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:43:10.484891 12674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:10.485208 12674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:10.485236 12674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:10.486622 12674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:10.486665 12674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:10.486691 12674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:43:10.488093 12674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:10.488363 12674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:10.488756 12674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:10.493856 12683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:10.493935 12682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:10.494112 12674 server_base.cc:1034] running on GCE node
W20260809 00:43:10.493862 12686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:10.494400 12674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:10.494688 12674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:10.495857 12674 hybrid_clock.cc:648] HybridClock initialized: now 1786236190495842 us; error 27 us; skew 500 ppm
I20260809 00:43:10.497255 12674 webserver.cc:460] Webserver started at http://127.30.104.194:39609/ using document root <none> and password file <none>
I20260809 00:43:10.497520 12674 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:10.497632 12674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:10.497778 12674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:10.498875 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "b88b37a74ec141a4a093557082fbd9f0"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "8e8e181daf8f77f95ac184ae62a7fd39"
server_key_iv: "3bbc0d81d43c199498279a2aeadb5332"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.499254 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "b88b37a74ec141a4a093557082fbd9f0"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "8e8e181daf8f77f95ac184ae62a7fd39"
server_key_iv: "3bbc0d81d43c199498279a2aeadb5332"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.500739 12674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:43:10.501550 12695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.501755 12674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:43:10.501858 12674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "b88b37a74ec141a4a093557082fbd9f0"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "8e8e181daf8f77f95ac184ae62a7fd39"
server_key_iv: "3bbc0d81d43c199498279a2aeadb5332"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.501941 12674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:10.505388 12674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:10.505656 12674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:10.505796 12674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:10.506000 12674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:10.506309 12674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:10.506361 12674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.506402 12674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:10.506429 12674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.512496 12674 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:35301
I20260809 00:43:10.512537 12867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:35301 every 8 connection(s)
I20260809 00:43:10.512903 12674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:43:10.516374 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.513750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:39439 (local address 127.30.104.254:36027)
0809 00:43:10.513840 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:10.513842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:10.514055 (+ 213us) server_negotiation.cc:407] Connection header received
0809 00:43:10.514629 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:10.514631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:10.514636 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:10.514701 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:10.516137 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:10.516138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:10.516139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:43:10.516143 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:10.516159 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:43:10.516164 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:10.516214 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:10.516282 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 00:43:10.516296 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 00:43:10.516601 12871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.513710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36027 (local address 127.30.104.194:39439)
0809 00:43:10.513932 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:10.513968 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:10.514445 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:10.514835 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:10.514839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:10.514847 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:10.515979 (+ 1132us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:43:10.515980 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:10.516010 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:43:10.516042 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:10.516219 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:10.516221 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:10.516223 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:43:10.516275 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:43:10.516326 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
I20260809 00:43:10.516582 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 12674
I20260809 00:43:10.516687 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:43:10.516861 31139 external_mini_cluster.cc:1428] Setting key a4a4323785a55dd370ebae84488dd713
I20260809 00:43:10.517174 12868 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36027
I20260809 00:43:10.517275 12868 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:10.517429 12868 heartbeater.cc:511] Master 127.30.104.254:36027 requested a full tablet report, sending...
I20260809 00:43:10.517876 12404 ts_manager.cc:194] Registered new tserver with Master: b88b37a74ec141a4a093557082fbd9f0 (127.30.104.194:35301)
I20260809 00:43:10.517962 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36027
--builtin_ntp_servers=127.30.104.212:37077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 00:43:10.602401 12872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:10.602622 12872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:10.602643 12872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:10.603776 12872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:10.603811 12872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:10.603837 12872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:43:10.605275 12872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:36027
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:10.605523 12872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:10.605834 12872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:10.611131 12881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:10.611151 12880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:10.611151 12883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:10.611589 12872 server_base.cc:1034] running on GCE node
I20260809 00:43:10.611837 12872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:10.612106 12872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:10.613283 12872 hybrid_clock.cc:648] HybridClock initialized: now 1786236190613268 us; error 34 us; skew 500 ppm
I20260809 00:43:10.614892 12872 webserver.cc:460] Webserver started at http://127.30.104.195:39247/ using document root <none> and password file <none>
I20260809 00:43:10.615154 12872 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:10.615204 12872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:10.615343 12872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:10.616523 12872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "18b5adb3f4a247f0886a893d09d988be"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "802ca9e0bbe3fe11518951307f72956f"
server_key_iv: "e9cded22e7b14809e1161b01ab8951ef"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.616909 12872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "18b5adb3f4a247f0886a893d09d988be"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "802ca9e0bbe3fe11518951307f72956f"
server_key_iv: "e9cded22e7b14809e1161b01ab8951ef"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.618322 12872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:43:10.619123 12892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.619318 12872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:43:10.619477 12872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "18b5adb3f4a247f0886a893d09d988be"
format_stamp: "Formatted at 2026-08-09 00:43:10 on dist-test-slave-c12x"
server_key: "802ca9e0bbe3fe11518951307f72956f"
server_key_iv: "e9cded22e7b14809e1161b01ab8951ef"
server_key_version: "encryptionkey@0"
I20260809 00:43:10.619570 12872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:10.622998 12872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:10.623302 12872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:10.623500 12872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:10.623735 12872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:10.624078 12872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:10.624145 12872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.624186 12872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:10.624213 12872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:10.630136 12872 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:46015
I20260809 00:43:10.630182 13065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:46015 every 8 connection(s)
I20260809 00:43:10.630542 12872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:43:10.633601 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.631379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:49785 (local address 127.30.104.254:36027)
0809 00:43:10.631467 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:10.631469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:10.631763 (+ 294us) server_negotiation.cc:407] Connection header received
0809 00:43:10.632215 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:10.632216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:10.632222 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:10.632280 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:10.633397 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:10.633399 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:10.633399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:43:10.633404 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:10.633422 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:43:10.633427 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:10.633483 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:10.633531 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:43:10.633544 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:43:10.633788 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:10.631372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36027 (local address 127.30.104.195:49785)
0809 00:43:10.631615 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:10.631650 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:10.632124 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:10.632333 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:10.632336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:10.632344 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:10.633241 (+ 897us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:43:10.633241 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:10.633269 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:43:10.633297 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:10.633480 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:10.633482 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:10.633485 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:43:10.633537 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:43:10.633586 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":136,"threads_started":1}
I20260809 00:43:10.633983 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 12872
I20260809 00:43:10.634096 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:43:10.634277 13066 heartbeater.cc:347] Connected to a master server at 127.30.104.254:36027
I20260809 00:43:10.634305 31139 external_mini_cluster.cc:1428] Setting key aa0683ca91c9d43b7ba37b1a5558bf45
I20260809 00:43:10.634377 13066 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:10.634541 13066 heartbeater.cc:511] Master 127.30.104.254:36027 requested a full tablet report, sending...
I20260809 00:43:10.635262 12404 ts_manager.cc:194] Registered new tserver with Master: 18b5adb3f4a247f0886a893d09d988be (127.30.104.195:46015)
I20260809 00:43:10.646612 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 00:43:10.648710 12461 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:43:10.648162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54536 (local address 127.30.104.254:36027)
0809 00:43:10.648231 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:10.648233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:10.648362 (+ 129us) server_negotiation.cc:407] Connection header received
0809 00:43:10.648407 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:10.648407 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:10.648412 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:10.648475 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:10.648611 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54536: BlockingRecv error: recv got EOF from 127.0.0.1:54536 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:43:10.648773 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 12484
I20260809 00:43:10.661126 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 12674
I20260809 00:43:10.677910 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 12872
I20260809 00:43:10.692480 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 12364
2026-08-09T00:43:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (596 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:43:10 dist-test-slave-c12x krb5kdc[13085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:43:10 dist-test-slave-c12x krb5kdc[13085](info): set up 2 sockets
Aug 09 00:43:10 dist-test-slave-c12x krb5kdc[13085](info): commencing operation
krb5kdc: starting...
W20260809 00:43:11.933576 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:43:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:43:12.064597 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33805
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:33805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:43:12.152813 13102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:12.153038 13102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:12.153060 13102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:12.154271 13102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:43:12.154321 13102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:12.154336 13102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:12.154349 13102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:43:12.154361 13102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:43:12.155973 13102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:33805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:12.156273 13102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:12.156530 13102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:12.161724 13111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:12.161725 13115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:12.161778 13112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:12.162163 13102 server_base.cc:1034] running on GCE node
I20260809 00:43:12.162444 13102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:12.162776 13102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:12.163947 13102 hybrid_clock.cc:648] HybridClock initialized: now 1786236192163930 us; error 29 us; skew 500 ppm
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:43:12.166472 13102 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:43:12.167014 13102 webserver.cc:460] Webserver started at http://127.30.104.254:39827/ using document root <none> and password file <none>
I20260809 00:43:12.167268 13102 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:12.167340 13102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:12.167526 13102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:12.168666 13102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "39e9e27bd5974312b41792cb9fc5fd01"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "187e1077c9506e5541c3deb011972104"
server_key_iv: "bfcc3ff1cccb8ecb63e2937ebba188d6"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.169070 13102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "39e9e27bd5974312b41792cb9fc5fd01"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "187e1077c9506e5541c3deb011972104"
server_key_iv: "bfcc3ff1cccb8ecb63e2937ebba188d6"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.170624 13102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:43:12.171411 13126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.171656 13102 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:43:12.171766 13102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "39e9e27bd5974312b41792cb9fc5fd01"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "187e1077c9506e5541c3deb011972104"
server_key_iv: "bfcc3ff1cccb8ecb63e2937ebba188d6"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.171854 13102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:12.189476 13102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:12.191978 13102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:12.192163 13102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:12.196511 13102 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:33805
I20260809 00:43:12.196552 13197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:33805 every 8 connection(s)
I20260809 00:43:12.196973 13102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:43:12.198024 13198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:43:12.200475 13198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01: Bootstrap starting.
I20260809 00:43:12.201107 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 13102
I20260809 00:43:12.201128 13198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01: Neither blocks nor log segments found. Creating new log.
I20260809 00:43:12.201228 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:43:12.201422 31139 external_mini_cluster.cc:1428] Setting key 32543a5de37a447f6be9f49a3bbd0b2e
I20260809 00:43:12.201434 13198 log.cc:824] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01: Log is configured to *not* fsync() on all Append() calls
I20260809 00:43:12.202333 13198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01: No bootstrap required, opened a new log
I20260809 00:43:12.205080 13198 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e9e27bd5974312b41792cb9fc5fd01" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33805 } }
I20260809 00:43:12.205277 13198 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:43:12.205322 13198 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e9e27bd5974312b41792cb9fc5fd01, State: Initialized, Role: FOLLOWER
I20260809 00:43:12.205545 13198 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e9e27bd5974312b41792cb9fc5fd01" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33805 } }
I20260809 00:43:12.205670 13198 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:43:12.205715 13198 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:43:12.205772 13198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:43:12.206490 13198 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e9e27bd5974312b41792cb9fc5fd01" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33805 } }
I20260809 00:43:12.206607 13198 leader_election.cc:302] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39e9e27bd5974312b41792cb9fc5fd01; no voters:
I20260809 00:43:12.206786 13198 leader_election.cc:288] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:43:12.206858 13207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:43:12.207084 13198 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:43:12.207078 13207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [term 1 LEADER]: Becoming Leader. State: Replica: 39e9e27bd5974312b41792cb9fc5fd01, State: Running, Role: LEADER
I20260809 00:43:12.207321 13207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e9e27bd5974312b41792cb9fc5fd01" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33805 } }
I20260809 00:43:12.207820 13208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39e9e27bd5974312b41792cb9fc5fd01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e9e27bd5974312b41792cb9fc5fd01" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33805 } } }
I20260809 00:43:12.207908 13208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [sys.catalog]: This master's current role is: LEADER
I20260809 00:43:12.208169 13210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39e9e27bd5974312b41792cb9fc5fd01. Latest consensus state: current_term: 1 leader_uuid: "39e9e27bd5974312b41792cb9fc5fd01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e9e27bd5974312b41792cb9fc5fd01" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33805 } } }
I20260809 00:43:12.208487 13210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01 [sys.catalog]: This master's current role is: LEADER
W20260809 00:43:12.209182 13227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:43:12.209247 13227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:43:12.209352 13218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:43:12.209522 13218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:43:12.211674 13218 catalog_manager.cc:1347] Generated new cluster ID: 5c5d6cdd3be24c1797065aea2acabe1a
I20260809 00:43:12.211731 13218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:43:12.214027 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.202039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54916 (local address 127.30.104.254:33805)
0809 00:43:12.202249 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:12.202259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:12.202291 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:43:12.202873 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:12.202882 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:12.202950 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:12.203110 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:12.203822 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.205281 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.206263 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.206952 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.207477 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.210229 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:12.210233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:12.210234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:43:12.210247 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:12.212241 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.212648 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.212652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.212652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.212748 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.212968 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.212969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.212970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.213240 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:12.213425 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:12.213510 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 00:43:12.213660 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
I20260809 00:43:12.227741 13218 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:43:12.228489 13218 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:43:12.241483 13218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01: Generated new TSK 0
I20260809 00:43:12.241734 13218 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:43:12.279232 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33805
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:43:12.366569 13235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:12.366824 13235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:12.366851 13235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:12.368144 13235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:12.368191 13235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:12.368220 13235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:43:12.369771 13235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:12.370002 13235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:12.370213 13235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:12.376520 13247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:12.376580 13245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:12.376541 13243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:12.376897 13235 server_base.cc:1034] running on GCE node
I20260809 00:43:12.377121 13235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:12.377395 13235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:12.378582 13235 hybrid_clock.cc:648] HybridClock initialized: now 1786236192378565 us; error 38 us; skew 500 ppm
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:43:12.381546 13235 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:43:12.382110 13235 webserver.cc:460] Webserver started at http://127.30.104.193:33039/ using document root <none> and password file <none>
I20260809 00:43:12.382355 13235 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:12.382427 13235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:12.382599 13235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:12.383806 13235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "34b0d9c876274caea2bcf503c9f4964b"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "917f3e516ff723fe4bd9acdf2e6b1e17"
server_key_iv: "7e8b5eafa4d38bd4c4eb7077c3f4ef2a"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.384193 13235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "34b0d9c876274caea2bcf503c9f4964b"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "917f3e516ff723fe4bd9acdf2e6b1e17"
server_key_iv: "7e8b5eafa4d38bd4c4eb7077c3f4ef2a"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.385739 13235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:43:12.386680 13256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.386881 13235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:43:12.386997 13235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "34b0d9c876274caea2bcf503c9f4964b"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "917f3e516ff723fe4bd9acdf2e6b1e17"
server_key_iv: "7e8b5eafa4d38bd4c4eb7077c3f4ef2a"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.387092 13235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:12.399710 13235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:12.402261 13235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:12.402467 13235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:12.402750 13235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:12.403119 13235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:12.403177 13235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.403220 13235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:12.403249 13235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.410269 13235 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:42037
I20260809 00:43:12.410311 13427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:42037 every 8 connection(s)
I20260809 00:43:12.410701 13235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:43:12.415560 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 13235
I20260809 00:43:12.415690 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:43:12.415874 31139 external_mini_cluster.cc:1428] Setting key bb55147b45dd09d461f386f50441343d
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:43:12.421640 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.411686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:38443 (local address 127.30.104.254:33805)
0809 00:43:12.411770 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:12.411773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:12.411989 (+ 216us) server_negotiation.cc:407] Connection header received
0809 00:43:12.412527 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:12.412528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:12.412538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:12.412624 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:12.413625 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.414137 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.415225 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.415606 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.415704 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.418207 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:12.418209 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:12.418210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:43:12.418219 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:12.420017 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.420536 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.420538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.420538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.420611 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.420901 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.420903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.420903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.421037 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:12.421132 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:12.421438 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 00:43:12.421540 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:43:12.421844 13430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.411650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33805 (local address 127.30.104.193:38443)
0809 00:43:12.411852 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:12.411891 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:12.412406 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:12.412686 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:12.412689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:12.413312 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:12.413528 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.413534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.414238 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.414239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.415129 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.415132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.415714 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.415715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.415862 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.416534 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:12.416544 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:12.418089 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:12.420160 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:43:12.420168 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:43:12.420171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:43:12.420443 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:43:12.420703 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:43:12.420704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:43:12.420704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:43:12.420827 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:43:12.421127 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:12.421135 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:12.421292 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 00:43:12.421404 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 00:43:12.421580 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260809 00:43:12.422502 13428 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33805
I20260809 00:43:12.422626 13428 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:12.422837 13428 heartbeater.cc:511] Master 127.30.104.254:33805 requested a full tablet report, sending...
I20260809 00:43:12.423545 13151 ts_manager.cc:194] Registered new tserver with Master: 34b0d9c876274caea2bcf503c9f4964b (127.30.104.193:42037)
I20260809 00:43:12.424190 13151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:38443
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:43:12.447247 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33805
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:43:12.540527 13433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:12.540776 13433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:12.540803 13433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:12.542029 13433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:12.542079 13433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:12.542107 13433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:43:12.543772 13433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:12.544025 13433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:12.544226 13433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:12.549717 13441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:12.549728 13440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:12.549743 13444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:12.550024 13433 server_base.cc:1034] running on GCE node
I20260809 00:43:12.550283 13433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:12.550559 13433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:12.551748 13433 hybrid_clock.cc:648] HybridClock initialized: now 1786236192551715 us; error 47 us; skew 500 ppm
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:43:12.554575 13433 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:43:12.555195 13433 webserver.cc:460] Webserver started at http://127.30.104.194:45357/ using document root <none> and password file <none>
I20260809 00:43:12.555536 13433 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:12.555589 13433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:12.555722 13433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:12.556876 13433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "0d8a51fd6df94214ae0c72de55185a26"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "411902691ac86bc5289dda8bff9b68a6"
server_key_iv: "9a92801000982dc5249a4c76d1a9abf9"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.557271 13433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "0d8a51fd6df94214ae0c72de55185a26"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "411902691ac86bc5289dda8bff9b68a6"
server_key_iv: "9a92801000982dc5249a4c76d1a9abf9"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.559335 13433 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 00:43:12.560390 13458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.560606 13433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:43:12.560724 13433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "0d8a51fd6df94214ae0c72de55185a26"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "411902691ac86bc5289dda8bff9b68a6"
server_key_iv: "9a92801000982dc5249a4c76d1a9abf9"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.560825 13433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:12.586102 13433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:12.588744 13433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:12.588925 13433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:12.589212 13433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:12.589572 13433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:12.589630 13433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.589695 13433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:12.589725 13433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.596939 13433 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:33429
I20260809 00:43:12.597030 13632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:33429 every 8 connection(s)
I20260809 00:43:12.597432 13433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:43:12.604429 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 13433
I20260809 00:43:12.604554 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:43:12.604786 31139 external_mini_cluster.cc:1428] Setting key 6b33284330e241ef02b7f0a1d5b1428c
I20260809 00:43:12.608222 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.598272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:46113 (local address 127.30.104.254:33805)
0809 00:43:12.598339 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:12.598341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:12.598614 (+ 273us) server_negotiation.cc:407] Connection header received
0809 00:43:12.599164 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:12.599166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:12.599178 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:12.599238 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:12.600292 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.600812 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.601837 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.602047 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.602144 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.604733 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:12.604735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:12.604736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:43:12.604752 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:12.606429 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.606985 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.606987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.606987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.607057 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.607421 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.607422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.607422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.607592 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:12.607671 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:12.607998 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 00:43:12.608091 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:43:12.608564 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.598258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33805 (local address 127.30.104.194:46113)
0809 00:43:12.598480 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:12.598518 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:12.599051 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:12.599313 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:12.599317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:12.599983 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:12.600186 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.600190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.600915 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.600916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.601739 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.601741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.602157 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.602158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.602292 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.602903 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:12.602912 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:12.604608 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:12.606542 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:43:12.606551 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:43:12.606553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:43:12.606891 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:43:12.607135 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:43:12.607136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:43:12.607136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:43:12.607333 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:43:12.607673 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:12.607681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:12.607857 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 00:43:12.607974 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 00:43:12.608161 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260809 00:43:12.609196 13633 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33805
I20260809 00:43:12.609311 13633 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:12.609540 13633 heartbeater.cc:511] Master 127.30.104.254:33805 requested a full tablet report, sending...
I20260809 00:43:12.610039 13151 ts_manager.cc:194] Registered new tserver with Master: 0d8a51fd6df94214ae0c72de55185a26 (127.30.104.194:33429)
I20260809 00:43:12.610620 13151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:46113
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:43:12.634352 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33805
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:43:12.721164 13639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:43:12.721393 13639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:43:12.721421 13639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:43:12.722579 13639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:43:12.722616 13639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:43:12.722644 13639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:43:12.724138 13639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:43:12.724349 13639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:43:12.724547 13639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:43:12.729784 13646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:43:12.729784 13648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:12.730053 13639 server_base.cc:1034] running on GCE node
W20260809 00:43:12.729825 13645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:43:12.730311 13639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:43:12.730563 13639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:43:12.731705 13639 hybrid_clock.cc:648] HybridClock initialized: now 1786236192731695 us; error 30 us; skew 500 ppm
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:43:12.734282 13639 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:43:12.734787 13639 webserver.cc:460] Webserver started at http://127.30.104.195:33061/ using document root <none> and password file <none>
I20260809 00:43:12.735036 13639 fs_manager.cc:359] Metadata directory not provided
I20260809 00:43:12.735109 13639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:43:12.735266 13639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:43:12.736410 13639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "e0b15c1c68e6416592f9aa062544fdd0"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "b40e1adaaf11868086914ed0928ad70e"
server_key_iv: "31663de3f84f102ec845cf27a9f57ea7"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.736793 13639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "e0b15c1c68e6416592f9aa062544fdd0"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "b40e1adaaf11868086914ed0928ad70e"
server_key_iv: "31663de3f84f102ec845cf27a9f57ea7"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.738209 13639 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:43:12.739034 13658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.739229 13639 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:43:12.739341 13639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "e0b15c1c68e6416592f9aa062544fdd0"
format_stamp: "Formatted at 2026-08-09 00:43:12 on dist-test-slave-c12x"
server_key: "b40e1adaaf11868086914ed0928ad70e"
server_key_iv: "31663de3f84f102ec845cf27a9f57ea7"
server_key_version: "encryptionkey@0"
I20260809 00:43:12.739418 13639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:43:12.755818 13639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:43:12.757820 13639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:43:12.758001 13639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:43:12.758267 13639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:43:12.758652 13639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:43:12.758690 13639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.758713 13639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:43:12.758726 13639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:43:12.765003 13639 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:40435
I20260809 00:43:12.765043 13828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:40435 every 8 connection(s)
I20260809 00:43:12.765410 13639 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:43:12.771318 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 13639
I20260809 00:43:12.771458 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:43:12.771667 31139 external_mini_cluster.cc:1428] Setting key 9e2430f0853bacaaacbb64fab8a0fd24
Aug 09 00:43:12 dist-test-slave-c12x krb5kdc[13085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236192, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:43:12.774991 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.766235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:37767 (local address 127.30.104.254:33805)
0809 00:43:12.766305 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:12.766308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:12.766523 (+ 215us) server_negotiation.cc:407] Connection header received
0809 00:43:12.767008 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:12.767009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:12.767020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:12.767075 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:12.768123 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.768634 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.769634 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.769837 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.769897 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.772402 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:12.772404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:12.772404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:43:12.772412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:12.773710 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.774081 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.774082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.774082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.774130 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.774424 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.774425 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.774425 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.774536 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:12.774609 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:12.774854 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 00:43:12.774924 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:43:12.775265 13834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.766213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33805 (local address 127.30.104.195:37767)
0809 00:43:12.766393 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:12.766430 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:12.766921 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:12.767137 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:12.767141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:12.767817 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:43:12.768018 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.768023 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.768720 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.768720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.769544 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.769546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.769918 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.769919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.770018 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.770590 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:43:12.770599 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:43:12.772332 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:43:12.773785 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:43:12.773791 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:43:12.773793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:43:12.774015 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:43:12.774230 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:43:12.774231 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:43:12.774231 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:43:12.774346 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:43:12.774599 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:43:12.774603 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:43:12.774736 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 00:43:12.774826 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 00:43:12.774967 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
I20260809 00:43:12.775789 13831 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33805
I20260809 00:43:12.775925 13831 heartbeater.cc:464] Registering TS with master...
I20260809 00:43:12.776121 13831 heartbeater.cc:511] Master 127.30.104.254:33805 requested a full tablet report, sending...
I20260809 00:43:12.776492 13151 ts_manager.cc:194] Registered new tserver with Master: e0b15c1c68e6416592f9aa062544fdd0 (127.30.104.195:40435)
I20260809 00:43:12.776837 13151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:37767
I20260809 00:43:12.783883 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:43:12.791831 13206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.785368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54932 (local address 127.30.104.254:33805)
0809 00:43:12.785463 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:12.785465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:12.785560 (+ 95us) server_negotiation.cc:407] Connection header received
0809 00:43:12.785643 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:12.785643 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:12.785661 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:12.785724 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:43:12.786313 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.786853 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.787680 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.787868 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.787945 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.789166 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:43:12.789167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:43:12.789167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:43:12.789175 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:43:12.790461 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.790873 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.790875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.790875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.790938 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:43:12.791190 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:43:12.791191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:43:12.791192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:43:12.791369 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:43:12.791516 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:12.791634 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 00:43:12.791717 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:43:12.793686 13151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:43:12.795269 13151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:43:12.802538 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.798045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36920 (local address 127.30.104.194:33429)
0809 00:43:12.798246 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:12.798250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:12.798596 (+ 346us) server_negotiation.cc:407] Connection header received
0809 00:43:12.798642 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:12.798645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:12.798661 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:12.798735 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:43:12.798897 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.800034 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.801603 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.802108 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.802206 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.802269 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:12.802385 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 00:43:12.802430 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":97,"threads_started":1}
I20260809 00:43:12.802817 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.797860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.30.104.195:40435)
0809 00:43:12.798015 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:12.798021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:12.798372 (+ 351us) server_negotiation.cc:407] Connection header received
0809 00:43:12.798563 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:12.798566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:12.798583 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:12.798653 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:43:12.798854 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.800034 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.801315 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.802146 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.802218 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.802268 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:12.802623 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 00:43:12.802737 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":81,"threads_started":1}
I20260809 00:43:12.803056 13848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.797766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:40435 (local address 127.0.0.1:33802)
0809 00:43:12.798244 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:12.798284 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:12.798372 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:12.798721 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:12.798722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:12.798724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:43:12.798783 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.798785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.800138 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.800139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.801208 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.801211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.802303 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.802304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.802375 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.802379 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:43:12.802423 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:43:12.802464 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":2,"thread_start_us":116,"threads_started":1}
I20260809 00:43:12.802577 13849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.797767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:33429 (local address 127.0.0.1:36920)
0809 00:43:12.798477 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:12.798486 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:12.798529 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:12.798744 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:12.798745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:12.798748 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:43:12.798812 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.798813 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.800173 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.800175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.801494 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.801498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.802202 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.802203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.802284 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.802299 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:43:12.802371 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 00:43:12.802429 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":48,"thread_start_us":75,"threads_started":1}
I20260809 00:43:12.803763 13847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.797767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:42037 (local address 127.0.0.1:59512)
0809 00:43:12.798308 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:12.798317 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:12.798372 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:12.798700 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:12.798703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:12.798706 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:43:12.798783 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.798786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.801438 (+ 2652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.801439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.802553 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:12.802556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.803529 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.803530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:12.803578 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.803581 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:43:12.803654 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:43:12.803694 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":3,"thread_start_us":105,"threads_started":1}
I20260809 00:43:12.803786 13854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:12.798033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59512 (local address 127.30.104.193:42037)
0809 00:43:12.798246 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:12.798250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:12.798447 (+ 197us) server_negotiation.cc:407] Connection header received
0809 00:43:12.798495 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:12.798498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:12.798515 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:12.798596 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:43:12.800178 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.801298 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.802649 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:12.803341 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:12.803407 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:12.803479 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:12.803670 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 00:43:12.803708 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":100,"threads_started":1}
I20260809 00:43:12.803936 13735 tablet_service.cc:1467] Processing CreateTablet for tablet 9af8f7b4aacc441f9024d735d502c291 (DEFAULT_TABLE table=test-table [id=50dcb7905e8e4843984f9022dc6f4308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:43:12.803936 13539 tablet_service.cc:1467] Processing CreateTablet for tablet 9af8f7b4aacc441f9024d735d502c291 (DEFAULT_TABLE table=test-table [id=50dcb7905e8e4843984f9022dc6f4308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:43:12.804317 13735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af8f7b4aacc441f9024d735d502c291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:43:12.804317 13539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af8f7b4aacc441f9024d735d502c291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:43:12.806283 13321 tablet_service.cc:1467] Processing CreateTablet for tablet 9af8f7b4aacc441f9024d735d502c291 (DEFAULT_TABLE table=test-table [id=50dcb7905e8e4843984f9022dc6f4308]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:43:12.806663 13321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af8f7b4aacc441f9024d735d502c291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:43:12.807909 13856 tablet_bootstrap.cc:492] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Bootstrap starting.
I20260809 00:43:12.808285 13857 tablet_bootstrap.cc:492] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Bootstrap starting.
I20260809 00:43:12.808877 13856 tablet_bootstrap.cc:654] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Neither blocks nor log segments found. Creating new log.
I20260809 00:43:12.808934 13857 tablet_bootstrap.cc:654] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Neither blocks nor log segments found. Creating new log.
I20260809 00:43:12.809576 13857 log.cc:824] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Log is configured to *not* fsync() on all Append() calls
I20260809 00:43:12.809671 13856 log.cc:824] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Log is configured to *not* fsync() on all Append() calls
I20260809 00:43:12.810326 13861 tablet_bootstrap.cc:492] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b: Bootstrap starting.
I20260809 00:43:12.810485 13857 tablet_bootstrap.cc:492] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: No bootstrap required, opened a new log
I20260809 00:43:12.810572 13857 ts_tablet_manager.cc:1397] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:43:12.810966 13861 tablet_bootstrap.cc:654] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b: Neither blocks nor log segments found. Creating new log.
I20260809 00:43:12.811605 13861 log.cc:824] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:43:12.811951 13856 tablet_bootstrap.cc:492] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: No bootstrap required, opened a new log
I20260809 00:43:12.812048 13856 ts_tablet_manager.cc:1397] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 00:43:12.812439 13861 tablet_bootstrap.cc:492] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b: No bootstrap required, opened a new log
I20260809 00:43:12.812516 13861 ts_tablet_manager.cc:1397] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:43:12.812474 13857 raft_consensus.cc:348] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:12.812704 13857 raft_consensus.cc:374] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:43:12.812752 13857 raft_consensus.cc:729] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0b15c1c68e6416592f9aa062544fdd0, State: Initialized, Role: FOLLOWER
I20260809 00:43:12.812875 13857 consensus_queue.cc:260] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:12.813220 13857 ts_tablet_manager.cc:1428] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:43:12.813345 13831 heartbeater.cc:503] Master 127.30.104.254:33805 was elected leader, sending a full tablet report...
I20260809 00:43:12.814091 13856 raft_consensus.cc:348] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:12.814270 13856 raft_consensus.cc:374] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:43:12.814307 13856 raft_consensus.cc:729] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d8a51fd6df94214ae0c72de55185a26, State: Initialized, Role: FOLLOWER
I20260809 00:43:12.814426 13856 consensus_queue.cc:260] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:12.814491 13861 raft_consensus.cc:348] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:12.814661 13861 raft_consensus.cc:374] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:43:12.814707 13861 raft_consensus.cc:729] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b0d9c876274caea2bcf503c9f4964b, State: Initialized, Role: FOLLOWER
I20260809 00:43:12.814752 13856 ts_tablet_manager.cc:1428] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:43:12.814800 13633 heartbeater.cc:503] Master 127.30.104.254:33805 was elected leader, sending a full tablet report...
I20260809 00:43:12.814847 13861 consensus_queue.cc:260] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:12.815158 13861 ts_tablet_manager.cc:1428] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:43:12.815209 13428 heartbeater.cc:503] Master 127.30.104.254:33805 was elected leader, sending a full tablet report...
W20260809 00:43:12.848125 13634 tablet.cc:2369] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:43:12.911566 13429 tablet.cc:2369] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:43:13.016109 13833 tablet.cc:2369] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:43:13.173707 13868 raft_consensus.cc:482] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:43:13.173837 13868 raft_consensus.cc:504] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:13.174322 13868 leader_election.cc:288] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34b0d9c876274caea2bcf503c9f4964b (127.30.104.193:42037), e0b15c1c68e6416592f9aa062544fdd0 (127.30.104.195:40435)
I20260809 00:43:13.177520 13854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:13.174653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:58629 (local address 127.30.104.193:42037)
0809 00:43:13.174709 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:13.174711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:13.174759 (+ 48us) server_negotiation.cc:407] Connection header received
0809 00:43:13.174875 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:13.174876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:13.174891 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:13.174959 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:43:13.175130 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.175745 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.176762 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.177132 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.177216 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:13.177226 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:13.177402 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 00:43:13.177447 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:43:13.177536 13850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:13.174820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:40073 (local address 127.30.104.195:40435)
0809 00:43:13.174877 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:13.174880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:13.174911 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:43:13.175032 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:13.175033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:13.175042 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:13.175102 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:43:13.175253 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.175824 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.176777 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.177164 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.177249 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:13.177257 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:13.177410 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:43:13.177447 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:43:13.177549 13871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:13.174499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:42037 (local address 127.30.104.194:58629)
0809 00:43:13.174680 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:13.174694 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:13.174791 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:13.174967 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:13.174968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:13.174970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:43:13.175056 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:13.175058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.175834 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.175835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:13.176649 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:13.176651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.177217 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.177218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:13.177311 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:13.177314 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:43:13.177381 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 00:43:13.177430 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":4,"thread_start_us":75,"threads_started":1}
I20260809 00:43:13.177554 13874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:13.174587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:40435 (local address 127.30.104.194:40073)
0809 00:43:13.174835 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:13.174849 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:43:13.174929 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:43:13.175109 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:43:13.175109 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:43:13.175111 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:43:13.175182 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:13.175183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.175909 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.175910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:13.176666 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:43:13.176668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.177249 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.177250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:43:13.177331 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:13.177334 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:43:13.177403 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 00:43:13.177447 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":62,"threads_started":1}
I20260809 00:43:13.178172 13360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9af8f7b4aacc441f9024d735d502c291" candidate_uuid: "0d8a51fd6df94214ae0c72de55185a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b0d9c876274caea2bcf503c9f4964b" is_pre_election: true
I20260809 00:43:13.178323 13360 raft_consensus.cc:2454] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d8a51fd6df94214ae0c72de55185a26 in term 0.
I20260809 00:43:13.178185 13762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9af8f7b4aacc441f9024d735d502c291" candidate_uuid: "0d8a51fd6df94214ae0c72de55185a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0b15c1c68e6416592f9aa062544fdd0" is_pre_election: true
I20260809 00:43:13.178539 13762 raft_consensus.cc:2454] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d8a51fd6df94214ae0c72de55185a26 in term 0.
I20260809 00:43:13.178579 13469 leader_election.cc:302] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d8a51fd6df94214ae0c72de55185a26, 34b0d9c876274caea2bcf503c9f4964b; no voters:
I20260809 00:43:13.178725 13868 raft_consensus.cc:2781] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:43:13.178817 13868 raft_consensus.cc:482] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:43:13.178838 13868 raft_consensus.cc:3037] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:43:13.179664 13868 raft_consensus.cc:504] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:13.179827 13868 leader_election.cc:288] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [CANDIDATE]: Term 1 election: Requested vote from peers 34b0d9c876274caea2bcf503c9f4964b (127.30.104.193:42037), e0b15c1c68e6416592f9aa062544fdd0 (127.30.104.195:40435)
I20260809 00:43:13.179999 13360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9af8f7b4aacc441f9024d735d502c291" candidate_uuid: "0d8a51fd6df94214ae0c72de55185a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34b0d9c876274caea2bcf503c9f4964b"
I20260809 00:43:13.180068 13762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9af8f7b4aacc441f9024d735d502c291" candidate_uuid: "0d8a51fd6df94214ae0c72de55185a26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0b15c1c68e6416592f9aa062544fdd0"
I20260809 00:43:13.180105 13360 raft_consensus.cc:3037] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:43:13.180141 13762 raft_consensus.cc:3037] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:43:13.181079 13360 raft_consensus.cc:2454] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d8a51fd6df94214ae0c72de55185a26 in term 1.
I20260809 00:43:13.181079 13762 raft_consensus.cc:2454] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d8a51fd6df94214ae0c72de55185a26 in term 1.
I20260809 00:43:13.181293 13469 leader_election.cc:302] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d8a51fd6df94214ae0c72de55185a26, 34b0d9c876274caea2bcf503c9f4964b; no voters:
I20260809 00:43:13.181403 13868 raft_consensus.cc:2781] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:43:13.181545 13868 raft_consensus.cc:686] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 1 LEADER]: Becoming Leader. State: Replica: 0d8a51fd6df94214ae0c72de55185a26, State: Running, Role: LEADER
I20260809 00:43:13.181653 13868 consensus_queue.cc:237] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } }
I20260809 00:43:13.182569 13151 catalog_manager.cc:5495] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d8a51fd6df94214ae0c72de55185a26 (127.30.104.194). New cstate: current_term: 1 leader_uuid: "0d8a51fd6df94214ae0c72de55185a26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d8a51fd6df94214ae0c72de55185a26" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33429 } health_report { overall_health: HEALTHY } } }
I20260809 00:43:13.244372 13855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:13.241806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36936 (local address 127.30.104.194:33429)
0809 00:43:13.241883 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:43:13.241890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:43:13.241900 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:43:13.241938 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:43:13.241938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:43:13.241959 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:43:13.242030 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:43:13.242183 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.242752 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.243604 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:43:13.243811 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:43:13.243875 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:43:13.244057 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:43:13.244171 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:43:13.244253 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:43:13.244288 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 00:43:13.244309 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:43:13.246461 13360 raft_consensus.cc:1261] T 9af8f7b4aacc441f9024d735d502c291 P 34b0d9c876274caea2bcf503c9f4964b [term 1 FOLLOWER]: Refusing update from remote peer 0d8a51fd6df94214ae0c72de55185a26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:43:13.246480 13762 raft_consensus.cc:1261] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 1 FOLLOWER]: Refusing update from remote peer 0d8a51fd6df94214ae0c72de55185a26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:43:13.246903 13868 consensus_queue.cc:1035] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34b0d9c876274caea2bcf503c9f4964b" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 42037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:43:13.247066 13875 consensus_queue.cc:1035] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0b15c1c68e6416592f9aa062544fdd0" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 40435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:43:13.249920 13887 mvcc.cc:205] Tried to move back new op lower bound from 7316423447534346240 to 7316423447272693760. Current Snapshot: MvccSnapshot[applied={T|T < 7316423447534346240}]
I20260809 00:43:13.250816 13886 mvcc.cc:205] Tried to move back new op lower bound from 7316423447534346240 to 7316423447272693760. Current Snapshot: MvccSnapshot[applied={T|T < 7316423447534346240}]
I20260809 00:43:13.252556 13150 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:43:13.252682 13150 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:43:13.253546 13150 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39e9e27bd5974312b41792cb9fc5fd01: Sending DeleteTablet for 3 replicas of tablet 9af8f7b4aacc441f9024d735d502c291
I20260809 00:43:13.253906 13321 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af8f7b4aacc441f9024d735d502c291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:43:13 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:59512
I20260809 00:43:13.253950 13735 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af8f7b4aacc441f9024d735d502c291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:43:13 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:33802
I20260809 00:43:13.254293 13899 tablet_replica.cc:330] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: stopping tablet replica
I20260809 00:43:13.254415 13899 raft_consensus.cc:2229] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:43:13.254455 13539 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af8f7b4aacc441f9024d735d502c291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:43:13 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:36920
I20260809 00:43:13.254523 13899 raft_consensus.cc:2258] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:43:13.254684 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 13235
I20260809 00:43:13.255249 13899 ts_tablet_manager.cc:1905] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:43:13.257295 13899 ts_tablet_manager.cc:1918] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:43:13.257376 13899 log.cc:1196] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/9af8f7b4aacc441f9024d735d502c291
I20260809 00:43:13.257635 13899 ts_tablet_manager.cc:1939] T 9af8f7b4aacc441f9024d735d502c291 P e0b15c1c68e6416592f9aa062544fdd0: Deleting consensus metadata
I20260809 00:43:13.258097 13127 catalog_manager.cc:4841] TS e0b15c1c68e6416592f9aa062544fdd0 (127.30.104.195:40435): tablet 9af8f7b4aacc441f9024d735d502c291 (table test-table [id=50dcb7905e8e4843984f9022dc6f4308]) successfully deleted
I20260809 00:43:13.262991 13900 tablet_replica.cc:330] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: stopping tablet replica
I20260809 00:43:13.263957 13900 raft_consensus.cc:2229] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:43:13.264138 13900 raft_consensus.cc:2258] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:43:13.264636 13900 ts_tablet_manager.cc:1905] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:43:13.266920 13900 ts_tablet_manager.cc:1918] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:43:13.267011 13900 log.cc:1196] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/9af8f7b4aacc441f9024d735d502c291
I20260809 00:43:13.267287 13900 ts_tablet_manager.cc:1939] T 9af8f7b4aacc441f9024d735d502c291 P 0d8a51fd6df94214ae0c72de55185a26: Deleting consensus metadata
I20260809 00:43:13.267859 13128 catalog_manager.cc:4841] TS 0d8a51fd6df94214ae0c72de55185a26 (127.30.104.194:33429): tablet 9af8f7b4aacc441f9024d735d502c291 (table test-table [id=50dcb7905e8e4843984f9022dc6f4308]) successfully deleted
W20260809 00:43:13.270505 13131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:42037 (error 108)
I20260809 00:43:13.270797 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 13433
I20260809 00:43:13.273134 13849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:43:13.270933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:42037 (local address 127.0.0.1:59526)
0809 00:43:13.273019 (+ 2086us) negotiation.cc:107] Waiting for socket to connect
0809 00:43:13.273084 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:42037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2052}
W20260809 00:43:13.277124 13131 catalog_manager.cc:4584] TS 34b0d9c876274caea2bcf503c9f4964b (127.30.104.193:42037): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9af8f7b4aacc441f9024d735d502c291: Network error: Client connection negotiation failed: client connection to 127.30.104.193:42037: connect: Connection refused (error 111)
I20260809 00:43:13.287611 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 13639
I20260809 00:43:13.296339 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 13102
2026-08-09T00:43:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2605 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:43:13 dist-test-slave-c12x krb5kdc[13911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:43:13 dist-test-slave-c12x krb5kdc[13911](info): set up 2 sockets
Aug 09 00:43:13 dist-test-slave-c12x krb5kdc[13911](info): commencing operation
krb5kdc: starting...
W20260809 00:43:14.535800 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:43:14 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:43:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:43:14Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.104.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.104.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/rangeradmin_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/rangeradmin_127.30.104.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.104.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.104.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/rangerlookup_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/rangerlookup_127.30.104.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.104.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.104.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/HTTP_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/HTTP_127.30.104.212@KRBTEST.COM.keytab.
I20260809 00:43:14.730567 31139 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxSoRG4/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/postgres -l logfile start
2026-08-09 00:43:17.355 UTC [13949] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 00:43:17.355 UTC [13949] LOG: listening on IPv4 address "127.30.104.212", port 51751
2026-08-09 00:43:17.359 UTC [13949] LOG: listening on Unix socket "/tmp/.s.PGSQL.51751"
2026-08-09 00:43:17.397 UTC [13954] LOG: database system was shut down at 2026-08-09 00:43:15 UTC
2026-08-09 00:43:17.401 UTC [13949] LOG: database system is ready to accept connections
I20260809 00:43:18.531527 31139 mini_postgres.cc:96] Postgres bound to 51751
2026-08-09 00:43:18.538 UTC [13963] FATAL: database "slave" does not exist
127.30.104.212:51751 - accepting connections
I20260809 00:43:18.539619 31139 mini_ranger.cc:161] Starting Ranger...
I20260809 00:43:18.556216 31139 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 00:43:19.051949 31139 mini_ranger.cc:87] Created ranger Postgres database
I20260809 00:43:19.052062 31139 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-admin
2026-08-09 00:43:19,345 [I] DB FLAVOR :POSTGRES
2026-08-09 00:43:19,346 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:43:19,346 [I] Checking connection..
2026-08-09 00:43:19,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 00:43:19,604 [I] Checking connection passed.
2026-08-09 00:43:19,604 [I] --------- Verifying version history table ---------
2026-08-09 00:43:19,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:43:19,882 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:43:19,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:43:20,134 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:43:20,134 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 00:43:20,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 00:43:20.405 UTC [14067] WARNING: there is no transaction in progress
2026-08-09 00:43:20,414 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 00:43:20,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:43:20,682 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 00:43:20,682 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:43:20,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 00:43:20,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 00:43:21,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-c12x.c.gcp-upstream.internal','N') ;"
2026-08-09 00:43:21,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:43:21,753 [I] Table x_portal_user does not exist in database ranger
2026-08-09 00:43:21,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:43:22,028 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 00:43:22,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:43:22,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:43:22,558 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 00:43:22,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 00:43:23.611 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.632 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.650 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.668 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.686 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.704 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.768 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.776 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.792 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.803 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.813 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.828 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.840 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.848 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.856 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:23.864 UTC [14233] WARNING: there is no transaction in progress
2026-08-09 00:43:24,576 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 00:43:24,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 00:43:24,865 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 00:43:24,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:43:25,115 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 00:43:25,116 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:43:25,402 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 00:43:25,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:43:25,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:43:25,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-c12x.c.gcp-upstream.internal';"
2026-08-09 00:43:26,229 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 00:43:26,230 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 00:43:26,501 [I] DB_PATCHES have already been applied
I20260809 00:43:26.506758 31139 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxSoRG4/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 00:43:26.506834 31139 mini_ranger.cc:193] Using host: 127.30.104.212
I20260809 00:43:26.510882 31139 mini_ranger.cc:227] Ranger admin URL: http://127.30.104.212:56065
Aug 09, 2026 12:43:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:43:26 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:43:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:43:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:43:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:43:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:43:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 00:43:27.862736 31139 mini_ranger.cc:160] Time spent starting Ranger: real 9.323s user 0.000s sys 0.012s
Aug 09, 2026 12:43:50 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:43:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 12:43:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:43:50 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:43:51 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 00:43:51.794533 31139 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.104.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.104.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/rangerkms_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/rangerkms_127.30.104.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.104.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.104.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.104.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/HTTP_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.104.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/HTTP_127.30.104.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 00:43:51 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236231, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 00:43:51.887593 31139 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 00:43:51.899708 31139 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 00:43:52.883098 31139 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 00:43:52.883177 31139 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms
I20260809 00:43:52.883190 31139 mini_ranger_kms.cc:227] Using postgres at 127.30.104.212:51751
2026-08-09 00:43:53,273 [I] DB FLAVOR :POSTGRES
2026-08-09 00:43:53,274 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:43:53,274 [I] Checking connection
2026-08-09 00:43:53,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 00:43:53,604 [I] connection success
2026-08-09 00:43:53,605 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 00:43:53,605 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 00:43:53,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 00:43:53,858 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 00:43:53,858 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:43:53,858 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 00:43:53,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:51751/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 00:43:54,162 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 00:43:54.720072 31139 mini_ranger_kms.cc:310] Created kms service
I20260809 00:43:54.944792 31139 mini_ranger_kms.cc:326] Created kudu user
I20260809 00:43:54.989074 31139 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 00:43:54 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236231, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:43:55.437515 31139 mini_ranger_kms.cc:384] Added ranger policy
I20260809 00:43:55.437655 31139 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 00:43:55.437677 31139 mini_ranger_kms.cc:240] Using host: 127.30.104.212
I20260809 00:43:55.441838 31139 mini_ranger_kms.cc:276] Ranger KMS PID: 14513
I20260809 00:43:55.441987 31139 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.104.212:42333
Aug 09, 2026 12:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:43:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:43:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:43:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 00:43:57 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786236237, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.104.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:43:58.143379 31139 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.256s user 0.010s sys 0.335s
I20260809 00:43:58.143693 31139 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 00:43:58.143778 31139 mini_ranger_kms.cc:401] 127.30.104.212:42333/kms/v1/keys
Aug 09 00:43:58 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236237, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.104.212@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
Aug 09 00:43:58 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236237, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.104.212@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
Aug 09 00:44:01 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:44:01.920135 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39577
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:39577
--ranger_config_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:42333/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:02.106688 14592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:02.107161 14592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:02.107300 14592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:02.109578 14592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:44:02.109709 14592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:02.109764 14592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:02.109822 14592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:44:02.109855 14592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:44:02.112751 14592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.104.212:42333/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:39577
--ranger_config_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:02.113281 14592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:02.113680 14592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:02.125517 14603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:02.125679 14592 server_base.cc:1034] running on GCE node
W20260809 00:44:02.125660 14607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:02.125540 14604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:02.126572 14592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:02.127198 14592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:02.128446 14592 hybrid_clock.cc:648] HybridClock initialized: now 1786236242128404 us; error 72 us; skew 500 ppm
Aug 09 00:44:02 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236242, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:02.134974 14592 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:44:02.135896 14592 webserver.cc:460] Webserver started at http://127.30.104.254:34463/ using document root <none> and password file <none>
I20260809 00:44:02.136305 14592 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:02.136430 14592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:02.136672 14592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:44:02 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236242, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:02.264212 14592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "ecaafe118ad7453680b73bd2a5811d17"
format_stamp: "Formatted at 2026-08-09 00:44:02 on dist-test-slave-c12x"
server_key: "d6fade4ba58d1685751b1fd526a3b644"
server_key_iv: "e0c68e64f4bc67e007f7b4072db4188c"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:02.264781 14592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "ecaafe118ad7453680b73bd2a5811d17"
format_stamp: "Formatted at 2026-08-09 00:44:02 on dist-test-slave-c12x"
server_key: "d6fade4ba58d1685751b1fd526a3b644"
server_key_iv: "e0c68e64f4bc67e007f7b4072db4188c"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:02.267314 14592 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:44:02.297299 14618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:02.297699 14592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:44:02.297849 14592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "ecaafe118ad7453680b73bd2a5811d17"
format_stamp: "Formatted at 2026-08-09 00:44:02 on dist-test-slave-c12x"
server_key: "d6fade4ba58d1685751b1fd526a3b644"
server_key_iv: "e0c68e64f4bc67e007f7b4072db4188c"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:02.297989 14592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:02.352524 14592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:02.355721 14592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:02.356024 14592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:02.368101 14592 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:39577
I20260809 00:44:02.368151 14697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:39577 every 8 connection(s)
I20260809 00:44:02.368947 14592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:44:02.369192 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 14592
I20260809 00:44:02.369386 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:44:02.369769 31139 external_mini_cluster.cc:1428] Setting key fcd0f4618fa73caf5f3135ff0c899c6e
I20260809 00:44:02.370635 14698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:02.378629 14698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17: Bootstrap starting.
I20260809 00:44:02.381012 14698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:02.381958 14698 log.cc:824] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:02.383957 14698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17: No bootstrap required, opened a new log
Aug 09 00:44:02 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:02.388870 14698 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaafe118ad7453680b73bd2a5811d17" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39577 } }
I20260809 00:44:02.389225 14698 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:02.389298 14698 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecaafe118ad7453680b73bd2a5811d17, State: Initialized, Role: FOLLOWER
I20260809 00:44:02.389649 14698 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaafe118ad7453680b73bd2a5811d17" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39577 } }
I20260809 00:44:02.389863 14698 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:44:02.389966 14698 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:44:02.390151 14698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:02.392074 14698 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaafe118ad7453680b73bd2a5811d17" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39577 } }
I20260809 00:44:02.392442 14698 leader_election.cc:302] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecaafe118ad7453680b73bd2a5811d17; no voters:
I20260809 00:44:02.392853 14698 leader_election.cc:288] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:44:02.392992 14704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:44:02.392904 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:02.371391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.30.104.254:39577)
0809 00:44:02.372014 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:02.372026 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:02.372106 (+ 80us) server_negotiation.cc:407] Connection header received
0809 00:44:02.373377 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:02.373415 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:02.373547 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:02.373830 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:02.376046 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:02.378449 (+ 2403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:02.380894 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:02.382076 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:02.382388 (+ 312us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:02.386493 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:02.386505 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:02.386506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:02.386534 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:02.389493 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:02.390505 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:02.390518 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:02.390518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:02.390708 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:02.391082 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:02.391083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:02.391083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:02.391537 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:02.391839 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:02.392076 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 00:44:02.392324 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":402,"threads_started":1}
I20260809 00:44:02.393396 14698 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:44:02.393581 14704 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [term 1 LEADER]: Becoming Leader. State: Replica: ecaafe118ad7453680b73bd2a5811d17, State: Running, Role: LEADER
I20260809 00:44:02.394006 14704 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaafe118ad7453680b73bd2a5811d17" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39577 } }
I20260809 00:44:02.396950 14706 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecaafe118ad7453680b73bd2a5811d17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaafe118ad7453680b73bd2a5811d17" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39577 } } }
I20260809 00:44:02.397014 14708 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecaafe118ad7453680b73bd2a5811d17. Latest consensus state: current_term: 1 leader_uuid: "ecaafe118ad7453680b73bd2a5811d17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaafe118ad7453680b73bd2a5811d17" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39577 } } }
I20260809 00:44:02.397225 14706 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:02.397310 14708 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:02.398365 14698 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 00:44:02.398669 14730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:44:02.399745 14730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:44:02.425434 14730 catalog_manager.cc:1347] Generated new cluster ID: 85e250abade94beaad5f7c06624cb446
I20260809 00:44:02.425597 14730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:44:02.437335 14730 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:44:02.438505 14730 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:44:02.451659 14730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17: Generated new TSK 0
I20260809 00:44:02.451988 14730 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:44:03.744098 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:42333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39577
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:03.900009 14769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:03.900364 14769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:03.900415 14769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:03.902586 14769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:03.902662 14769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:03.902705 14769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:44:03.905267 14769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.104.212:42333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:03.905884 14769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:03.906436 14769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:03.915045 14776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:03.915203 14780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:03.915395 14777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:03.915570 14769 server_base.cc:1034] running on GCE node
I20260809 00:44:03.915788 14769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:03.916090 14769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:03.917460 14769 hybrid_clock.cc:648] HybridClock initialized: now 1786236243917428 us; error 46 us; skew 500 ppm
Aug 09 00:44:03 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236243, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:03.921618 14769 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:44:03.922228 14769 webserver.cc:460] Webserver started at http://127.30.104.193:38089/ using document root <none> and password file <none>
I20260809 00:44:03.922572 14769 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:03.922691 14769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:03.922857 14769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:44:03 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236243, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:03.938330 14769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "315061458efb4c0b90dd35644866f7ac"
format_stamp: "Formatted at 2026-08-09 00:44:03 on dist-test-slave-c12x"
server_key: "b8f8fd800142507abe7525c7744573b5"
server_key_iv: "fab3fe0a135aae1109d798932332ae46"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:03.938962 14769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "315061458efb4c0b90dd35644866f7ac"
format_stamp: "Formatted at 2026-08-09 00:44:03 on dist-test-slave-c12x"
server_key: "b8f8fd800142507abe7525c7744573b5"
server_key_iv: "fab3fe0a135aae1109d798932332ae46"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:03.944316 14769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 00:44:03.959623 14788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:03.959995 14769 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 00:44:03.960158 14769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "315061458efb4c0b90dd35644866f7ac"
format_stamp: "Formatted at 2026-08-09 00:44:03 on dist-test-slave-c12x"
server_key: "b8f8fd800142507abe7525c7744573b5"
server_key_iv: "fab3fe0a135aae1109d798932332ae46"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:03.960286 14769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:03.977715 14769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:03.980592 14769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:03.980885 14769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:03.981298 14769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:03.981772 14769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:03.981830 14769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:03.981865 14769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:03.981892 14769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:03.994794 14769 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:45431
I20260809 00:44:03.995376 14769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:44:03.997107 14973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:45431 every 8 connection(s)
I20260809 00:44:04.000560 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 14769
I20260809 00:44:04.000699 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:44:04.000926 31139 external_mini_cluster.cc:1428] Setting key 92d2d7aa2b687a50945f0fed5e6f599f
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236243, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:04.011912 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:03.997926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39577 (local address 127.30.104.193:52841)
0809 00:44:03.998824 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:03.998877 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:03.999843 (+ 966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:04.000312 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:04.000317 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:04.000763 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:04.001192 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.001200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.002208 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.002209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.003774 (+ 1565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.003778 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.004732 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.004733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.004964 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.005695 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:04.005706 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:04.007871 (+ 2165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:04.010179 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:04.010186 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:04.010189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:04.010502 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:04.010725 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:04.010726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:04.010727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:04.010858 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:04.011178 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:04.011185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:04.011357 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:44:04.011480 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 00:44:04.011653 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"thread_start_us":168,"threads_started":1}
I20260809 00:44:04.012526 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:03.998339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:52841 (local address 127.30.104.254:39577)
0809 00:44:03.999541 (+ 1202us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:03.999543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:03.999559 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:44:03.999979 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:03.999981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:03.999997 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.000182 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:04.001327 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.002089 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.003980 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.004408 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.004636 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.007987 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:04.007989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:04.007989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:04.007997 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:04.010082 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:04.010590 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:04.010592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:04.010592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:04.010659 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:04.010926 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:04.010927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:04.010927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:04.011087 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:04.011149 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.012328 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.012422 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1158,"thread_start_us":144,"threads_started":1}
I20260809 00:44:04.013450 14974 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39577
I20260809 00:44:04.013559 14974 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:04.013773 14974 heartbeater.cc:511] Master 127.30.104.254:39577 requested a full tablet report, sending...
I20260809 00:44:04.014333 14648 ts_manager.cc:194] Registered new tserver with Master: 315061458efb4c0b90dd35644866f7ac (127.30.104.193:45431)
I20260809 00:44:04.015249 14648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:52841
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:44:04.048022 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:42333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39577
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:04.194792 14987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:04.195111 14987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:04.195158 14987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:04.197435 14987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:04.197552 14987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:04.197608 14987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:44:04.200592 14987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.104.212:42333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:04.200915 14987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:04.201310 14987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:04.209496 15000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:04.209961 14996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:04.210299 14995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:04.211719 14987 server_base.cc:1034] running on GCE node
I20260809 00:44:04.212013 14987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:04.212352 14987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:04.213869 14987 hybrid_clock.cc:648] HybridClock initialized: now 1786236244213834 us; error 48 us; skew 500 ppm
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236244, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:04.232333 14987 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:44:04.233109 14987 webserver.cc:460] Webserver started at http://127.30.104.194:33487/ using document root <none> and password file <none>
I20260809 00:44:04.233489 14987 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:04.233623 14987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:04.233819 14987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236244, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:04.254932 14987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "cc349c8fb211484282eda1063d12e18a"
format_stamp: "Formatted at 2026-08-09 00:44:04 on dist-test-slave-c12x"
server_key: "f026fb5473c2fc3b1fdf28609c9c3f63"
server_key_iv: "61b9bcadfdaf7c2cbdfe21de347efe9a"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:04.255690 14987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "cc349c8fb211484282eda1063d12e18a"
format_stamp: "Formatted at 2026-08-09 00:44:04 on dist-test-slave-c12x"
server_key: "f026fb5473c2fc3b1fdf28609c9c3f63"
server_key_iv: "61b9bcadfdaf7c2cbdfe21de347efe9a"
server_key_version: "kuduclusterkey@0"
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786236244, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:04.258636 14987 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260809 00:44:04.279714 15007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:04.280035 14987 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.002s
I20260809 00:44:04.280172 14987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "cc349c8fb211484282eda1063d12e18a"
format_stamp: "Formatted at 2026-08-09 00:44:04 on dist-test-slave-c12x"
server_key: "f026fb5473c2fc3b1fdf28609c9c3f63"
server_key_iv: "61b9bcadfdaf7c2cbdfe21de347efe9a"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:04.280292 14987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:04.302762 14987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:04.307636 14987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:04.307891 14987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:04.308281 14987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:04.308800 14987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:04.308913 14987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:04.308992 14987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:04.309056 14987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:04.320700 14987 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:34797
I20260809 00:44:04.321018 15173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:34797 every 8 connection(s)
I20260809 00:44:04.321410 14987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:44:04.326581 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 14987
I20260809 00:44:04.326694 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:44:04.326917 31139 external_mini_cluster.cc:1428] Setting key da0cd17e59e8d61135f5024ab6b61549
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236244, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:04.335297 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.322864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:33661 (local address 127.30.104.254:39577)
0809 00:44:04.322939 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.322941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.323178 (+ 237us) server_negotiation.cc:407] Connection header received
0809 00:44:04.323948 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.323950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.323965 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.324026 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:04.324834 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.325352 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.326561 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.326890 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.326938 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.331621 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:04.331622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:04.331622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:04.331631 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:04.333532 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:04.334050 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:04.334051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:04.334051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:04.334132 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:04.334483 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:04.334484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:04.334484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:04.334647 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:04.334704 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.335087 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.335188 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:44:04.335970 15176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.322759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39577 (local address 127.30.104.194:33661)
0809 00:44:04.323030 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:04.323074 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:04.323834 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:04.324102 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:04.324105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:04.324421 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:04.324722 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.324727 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.325454 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.325455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.326440 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.326444 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.327050 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.327051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.327256 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.327934 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:04.327947 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:04.331482 (+ 3535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:04.333645 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:04.333654 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:04.333657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:04.333968 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:04.334281 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:04.334282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:04.334282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:04.334417 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:04.334755 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:04.334761 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:04.334940 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 00:44:04.335471 (+ 531us) client_negotiation.cc:240] Negotiation successful
0809 00:44:04.335673 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":160,"threads_started":1}
I20260809 00:44:04.336612 15174 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39577
I20260809 00:44:04.336720 15174 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:04.336932 15174 heartbeater.cc:511] Master 127.30.104.254:39577 requested a full tablet report, sending...
I20260809 00:44:04.337319 14648 ts_manager.cc:194] Registered new tserver with Master: cc349c8fb211484282eda1063d12e18a (127.30.104.194:34797)
I20260809 00:44:04.337863 14648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:33661
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:44:04.358644 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:42333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39577
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:04.513180 15182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:04.513499 15182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:04.513540 15182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:04.515390 15182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:04.515481 15182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:04.515530 15182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:44:04.518075 15182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.104.212:42333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:04.518354 15182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:04.518679 15182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:04.527500 15194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:04.527742 15190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:04.527971 15182 server_base.cc:1034] running on GCE node
W20260809 00:44:04.527989 15191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:04.528323 15182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:04.528671 15182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:04.529860 15182 hybrid_clock.cc:648] HybridClock initialized: now 1786236244529842 us; error 59 us; skew 500 ppm
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236244, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:04.534242 15182 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:44:04.534883 15182 webserver.cc:460] Webserver started at http://127.30.104.195:40413/ using document root <none> and password file <none>
I20260809 00:44:04.535202 15182 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:04.535324 15182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:04.535557 15182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236244, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:04.553584 15182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf"
format_stamp: "Formatted at 2026-08-09 00:44:04 on dist-test-slave-c12x"
server_key: "b76d6510958f88cd200c341e80c4609a"
server_key_iv: "0930bc7a3185bb43d631cd841400d491"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:04.554798 15182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf"
format_stamp: "Formatted at 2026-08-09 00:44:04 on dist-test-slave-c12x"
server_key: "b76d6510958f88cd200c341e80c4609a"
server_key_iv: "0930bc7a3185bb43d631cd841400d491"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:04.557139 15182 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 00:44:04.578646 15203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:04.578991 15182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 00:44:04.579113 15182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf"
format_stamp: "Formatted at 2026-08-09 00:44:04 on dist-test-slave-c12x"
server_key: "b76d6510958f88cd200c341e80c4609a"
server_key_iv: "0930bc7a3185bb43d631cd841400d491"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:04.579249 15182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:04.607971 15182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:04.611351 15182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:04.611567 15182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:04.611879 15182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:04.612270 15182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:04.612327 15182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:04.612371 15182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:04.612396 15182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:04.631381 15182 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:43661
I20260809 00:44:04.631657 15375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:43661 every 8 connection(s)
I20260809 00:44:04.631968 15182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:44:04.640424 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 15182
I20260809 00:44:04.640568 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:44:04.640772 31139 external_mini_cluster.cc:1428] Setting key 9d474f3abfa5a2e70a261e34aaee4ab0
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236244, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:04.645336 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.632896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:40741 (local address 127.30.104.254:39577)
0809 00:44:04.632981 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.632983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.633301 (+ 318us) server_negotiation.cc:407] Connection header received
0809 00:44:04.634363 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.634365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.634383 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.634475 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:04.636064 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.636595 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.638535 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.638743 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.638813 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.641833 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:04.641834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:04.641835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:04.641844 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:04.643722 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:04.644153 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:04.644154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:04.644154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:04.644221 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:04.644540 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:04.644541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:04.644541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:04.644746 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:04.644825 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.645122 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.645235 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:44:04.645531 15378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.632831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39577 (local address 127.30.104.195:40741)
0809 00:44:04.633123 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:04.633171 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:04.634104 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:04.634587 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:04.634592 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:04.635411 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:04.635916 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.635925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.636726 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.636727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.638415 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.638420 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.638872 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.638874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.639127 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.639910 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:04.639922 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:04.641714 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:04.643820 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:04.643827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:04.643829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:04.644075 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:04.644339 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:04.644340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:04.644340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:04.644463 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:04.644823 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:04.644829 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:04.644981 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 00:44:04.645074 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:44:04.645205 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":133,"threads_started":1}
I20260809 00:44:04.646149 15376 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39577
I20260809 00:44:04.646296 15376 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:04.646595 15376 heartbeater.cc:511] Master 127.30.104.254:39577 requested a full tablet report, sending...
I20260809 00:44:04.647236 14648 ts_manager.cc:194] Registered new tserver with Master: bcdb7bdb1eff4a8e83dc3b82017808bf (127.30.104.195:43661)
I20260809 00:44:04.647878 14648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:40741
I20260809 00:44:04.653393 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:44:04.665758 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.654998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46696 (local address 127.30.104.254:39577)
0809 00:44:04.655086 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.655089 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.655920 (+ 831us) server_negotiation.cc:407] Connection header received
0809 00:44:04.656040 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.656041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.656050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.656106 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:04.657350 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.657848 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.659596 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.659798 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.659863 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.662277 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:04.662278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:04.662279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:04.662288 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:04.663871 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:04.664497 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:04.664498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:04.664498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:04.664565 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:04.664888 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:04.664889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:04.664889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:04.665055 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:04.665176 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.665400 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.665569 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:44:04.668280 14648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:44:04.672875 14648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:44:04.681740 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.677725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46310 (local address 127.30.104.195:43661)
0809 00:44:04.677934 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.677940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.677950 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:44:04.678027 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.678029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.678046 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.678131 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:04.678291 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.679188 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.680662 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.681252 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.681395 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.681468 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.681562 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.681606 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260809 00:44:04.681703 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.677105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:43661 (local address 127.0.0.1:46310)
0809 00:44:04.677511 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:04.677557 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:04.677647 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:04.678134 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:04.678137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:04.678140 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:04.678223 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.678226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.679275 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.679277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.680499 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.680502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.681347 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.681348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.681443 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.681457 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:44:04.681548 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 00:44:04.681606 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":311,"threads_started":1}
I20260809 00:44:04.683110 15278 tablet_service.cc:1467] Processing CreateTablet for tablet d9a1500cb289435cb50c6985ee3fa2f2 (DEFAULT_TABLE table=test-table [id=07b7717f5946487080a08b46755af649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:44:04.683624 15278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a1500cb289435cb50c6985ee3fa2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:04.685415 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.677123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45431 (local address 127.0.0.1:35976)
0809 00:44:04.677813 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:04.677823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:04.677877 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:04.678561 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:04.678562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:04.678565 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:04.678607 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.678609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.681577 (+ 2968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.681578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.683312 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.683316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.685084 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.685086 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.685163 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.685168 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:44:04.685227 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 00:44:04.685288 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":83,"threads_started":1}
I20260809 00:44:04.686045 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.677530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35976 (local address 127.30.104.193:45431)
0809 00:44:04.678254 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.678260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.678282 (+ 22us) server_negotiation.cc:407] Connection header received
0809 00:44:04.678375 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.678377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.678408 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.678570 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:04.678678 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.681374 (+ 2696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.683487 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.684755 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.684890 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.684999 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.685801 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.685929 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":71,"threads_started":1}
I20260809 00:44:04.687182 15391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.677217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:34797 (local address 127.0.0.1:37384)
0809 00:44:04.682214 (+ 4997us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:04.682228 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:04.682283 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:04.682666 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:04.682668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:04.682669 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:04.682702 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.682703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.684245 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.684246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.685433 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.685436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.686913 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.686914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.686978 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.686981 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:44:04.687037 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:44:04.687098 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4981,"thread_start_us":216,"threads_started":1}
I20260809 00:44:04.687448 14851 tablet_service.cc:1467] Processing CreateTablet for tablet d9a1500cb289435cb50c6985ee3fa2f2 (DEFAULT_TABLE table=test-table [id=07b7717f5946487080a08b46755af649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:44:04.687800 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.677641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.30.104.194:34797)
0809 00:44:04.679619 (+ 1978us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.679624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.682357 (+ 2733us) server_negotiation.cc:407] Connection header received
0809 00:44:04.682412 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.682414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.682430 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.682525 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:04.682779 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.684126 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.685640 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.686677 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.686813 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.686875 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.687568 (+ 693us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.687678 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1900,"thread_start_us":98,"threads_started":1}
I20260809 00:44:04.687860 14851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a1500cb289435cb50c6985ee3fa2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:04.692463 15400 tablet_bootstrap.cc:492] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac: Bootstrap starting.
I20260809 00:44:04.693773 15400 tablet_bootstrap.cc:654] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:04.694504 15400 log.cc:824] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:04.695509 15400 tablet_bootstrap.cc:492] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac: No bootstrap required, opened a new log
I20260809 00:44:04.695585 15400 ts_tablet_manager.cc:1397] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:44:04.700248 15400 raft_consensus.cc:348] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.700500 15400 raft_consensus.cc:374] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:04.700537 15400 raft_consensus.cc:729] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 315061458efb4c0b90dd35644866f7ac, State: Initialized, Role: FOLLOWER
I20260809 00:44:04.700708 15400 consensus_queue.cc:260] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.701047 15400 ts_tablet_manager.cc:1428] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260809 00:44:04.701186 14974 heartbeater.cc:503] Master 127.30.104.254:39577 was elected leader, sending a full tablet report...
I20260809 00:44:04.701771 15078 tablet_service.cc:1467] Processing CreateTablet for tablet d9a1500cb289435cb50c6985ee3fa2f2 (DEFAULT_TABLE table=test-table [id=07b7717f5946487080a08b46755af649]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:44:04.702477 15078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9a1500cb289435cb50c6985ee3fa2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:04.708896 15404 tablet_bootstrap.cc:492] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Bootstrap starting.
I20260809 00:44:04.710728 15404 tablet_bootstrap.cc:654] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:04.711536 15402 tablet_bootstrap.cc:492] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Bootstrap starting.
I20260809 00:44:04.713732 15402 tablet_bootstrap.cc:654] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:04.714368 15402 log.cc:824] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:04.715355 15404 log.cc:824] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:04.716543 15402 tablet_bootstrap.cc:492] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: No bootstrap required, opened a new log
I20260809 00:44:04.716687 15404 tablet_bootstrap.cc:492] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: No bootstrap required, opened a new log
I20260809 00:44:04.716684 15402 ts_tablet_manager.cc:1397] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 00:44:04.716845 15404 ts_tablet_manager.cc:1397] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260809 00:44:04.721144 15402 raft_consensus.cc:348] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.721475 15402 raft_consensus.cc:374] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:04.721592 15402 raft_consensus.cc:729] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcdb7bdb1eff4a8e83dc3b82017808bf, State: Initialized, Role: FOLLOWER
I20260809 00:44:04.721491 15404 raft_consensus.cc:348] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.721648 15404 raft_consensus.cc:374] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:04.721689 15404 raft_consensus.cc:729] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc349c8fb211484282eda1063d12e18a, State: Initialized, Role: FOLLOWER
I20260809 00:44:04.721848 15404 consensus_queue.cc:260] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.721848 15402 consensus_queue.cc:260] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.722368 15402 ts_tablet_manager.cc:1428] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260809 00:44:04.722368 15404 ts_tablet_manager.cc:1428] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 00:44:04.722532 15174 heartbeater.cc:503] Master 127.30.104.254:39577 was elected leader, sending a full tablet report...
I20260809 00:44:04.722559 15376 heartbeater.cc:503] Master 127.30.104.254:39577 was elected leader, sending a full tablet report...
W20260809 00:44:04.749328 14975 tablet.cc:2369] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:44:04.823184 15175 tablet.cc:2369] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 00:44:04 dist-test-slave-c12x krb5kdc[13911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236244, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
W20260809 00:44:04.882822 15377 tablet.cc:2369] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:44:04.911716 15407 raft_consensus.cc:482] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:44:04.911868 15407 raft_consensus.cc:504] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.912341 15407 leader_election.cc:288] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcdb7bdb1eff4a8e83dc3b82017808bf (127.30.104.195:43661), 315061458efb4c0b90dd35644866f7ac (127.30.104.193:45431)
I20260809 00:44:04.918629 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.913051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:33115 (local address 127.30.104.195:43661)
0809 00:44:04.913113 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.913115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.913125 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:44:04.913155 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.913156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.913170 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.913221 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:04.914007 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.914901 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.917419 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.918173 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.918240 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.918251 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.918496 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.918544 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:44:04.919343 15412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.912536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:43661 (local address 127.30.104.194:33115)
0809 00:44:04.912880 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:04.912896 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:04.912954 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:04.913759 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:04.913760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:04.913763 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:04.913890 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.913892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.915272 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.915273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.917229 (+ 1956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.917234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.918327 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.918328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.918413 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.918417 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:44:04.919194 (+ 777us) client_negotiation.cc:240] Negotiation successful
0809 00:44:04.919255 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":66,"threads_started":1}
I20260809 00:44:04.919940 15306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9a1500cb289435cb50c6985ee3fa2f2" candidate_uuid: "cc349c8fb211484282eda1063d12e18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" is_pre_election: true
I20260809 00:44:04.920228 15306 raft_consensus.cc:2454] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc349c8fb211484282eda1063d12e18a in term 0.
I20260809 00:44:04.920615 15009 leader_election.cc:302] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcdb7bdb1eff4a8e83dc3b82017808bf, cc349c8fb211484282eda1063d12e18a; no voters:
I20260809 00:44:04.920792 15407 raft_consensus.cc:2781] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:44:04.920843 15407 raft_consensus.cc:482] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:44:04.920838 15394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.913520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:56343 (local address 127.30.104.193:45431)
0809 00:44:04.913583 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.913586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.913596 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:44:04.913627 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.913628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.913639 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.913693 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:04.914514 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.915184 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.918858 (+ 3674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.919790 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.919852 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.919861 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.920706 (+ 845us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.920764 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:44:04.920876 15407 raft_consensus.cc:3037] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:04.922318 15407 raft_consensus.cc:504] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.922549 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.912735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45431 (local address 127.30.104.194:56343)
0809 00:44:04.913450 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:04.913463 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:04.913544 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:04.914105 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:04.914109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:04.914113 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:04.914392 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.914396 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.915714 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.915714 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.918365 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:04.918368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.920407 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.920407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:04.920586 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.920591 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:44:04.922174 (+ 1583us) client_negotiation.cc:240] Negotiation successful
0809 00:44:04.922235 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":20,"thread_start_us":71,"threads_started":1}
I20260809 00:44:04.922925 14897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9a1500cb289435cb50c6985ee3fa2f2" candidate_uuid: "cc349c8fb211484282eda1063d12e18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315061458efb4c0b90dd35644866f7ac"
I20260809 00:44:04.923079 14897 raft_consensus.cc:3037] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:04.922986 14895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9a1500cb289435cb50c6985ee3fa2f2" candidate_uuid: "cc349c8fb211484282eda1063d12e18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "315061458efb4c0b90dd35644866f7ac" is_pre_election: true
I20260809 00:44:04.924571 14897 raft_consensus.cc:2454] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc349c8fb211484282eda1063d12e18a in term 1.
I20260809 00:44:04.924819 15012 leader_election.cc:302] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 315061458efb4c0b90dd35644866f7ac, cc349c8fb211484282eda1063d12e18a; no voters:
I20260809 00:44:04.922570 15407 leader_election.cc:288] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [CANDIDATE]: Term 1 election: Requested vote from peers bcdb7bdb1eff4a8e83dc3b82017808bf (127.30.104.195:43661), 315061458efb4c0b90dd35644866f7ac (127.30.104.193:45431)
I20260809 00:44:04.925285 15407 raft_consensus.cc:2781] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:44:04.925324 15407 raft_consensus.cc:686] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 1 LEADER]: Becoming Leader. State: Replica: cc349c8fb211484282eda1063d12e18a, State: Running, Role: LEADER
I20260809 00:44:04.925257 15306 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9a1500cb289435cb50c6985ee3fa2f2" candidate_uuid: "cc349c8fb211484282eda1063d12e18a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf"
I20260809 00:44:04.925369 15306 raft_consensus.cc:3037] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:04.925414 15407 consensus_queue.cc:237] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } }
I20260809 00:44:04.926534 14648 catalog_manager.cc:5495] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a reported cstate change: term changed from 0 to 1, leader changed from <none> to cc349c8fb211484282eda1063d12e18a (127.30.104.194). New cstate: current_term: 1 leader_uuid: "cc349c8fb211484282eda1063d12e18a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc349c8fb211484282eda1063d12e18a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 } health_report { overall_health: UNKNOWN } } }
I20260809 00:44:04.926785 15306 raft_consensus.cc:2454] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc349c8fb211484282eda1063d12e18a in term 1.
I20260809 00:44:04.972779 15397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:04.968896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37392 (local address 127.30.104.194:34797)
0809 00:44:04.968942 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:04.968945 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:04.969002 (+ 57us) server_negotiation.cc:407] Connection header received
0809 00:44:04.969090 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:04.969091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:04.969125 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:04.969204 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:44:04.969468 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.970288 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.971398 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:04.971708 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:04.971788 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:04.972043 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:44:04.972187 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:44:04.972268 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:04.972531 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 00:44:04.972572 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:44:04.975344 14897 raft_consensus.cc:1261] T d9a1500cb289435cb50c6985ee3fa2f2 P 315061458efb4c0b90dd35644866f7ac [term 1 FOLLOWER]: Refusing update from remote peer cc349c8fb211484282eda1063d12e18a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:44:04.975684 15306 raft_consensus.cc:1261] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 1 FOLLOWER]: Refusing update from remote peer cc349c8fb211484282eda1063d12e18a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:44:04.975765 15415 consensus_queue.cc:1035] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [LEADER]: Connected to new peer: Peer: permanent_uuid: "315061458efb4c0b90dd35644866f7ac" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:44:04.975992 15415 consensus_queue.cc:1035] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcdb7bdb1eff4a8e83dc3b82017808bf" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:44:04.980582 15419 mvcc.cc:205] Tried to move back new op lower bound from 7316423659415621632 to 7316423659215208448. Current Snapshot: MvccSnapshot[applied={T|T < 7316423659415621632}]
I20260809 00:44:04.983667 15422 mvcc.cc:205] Tried to move back new op lower bound from 7316423659415621632 to 7316423659215208448. Current Snapshot: MvccSnapshot[applied={T|T < 7316423659415621632}]
I20260809 00:44:04.984413 15421 mvcc.cc:205] Tried to move back new op lower bound from 7316423659415621632 to 7316423659215208448. Current Snapshot: MvccSnapshot[applied={T|T < 7316423659415621632}]
I20260809 00:44:04.987349 14648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:44:04.987493 14648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:44:04.988245 14648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecaafe118ad7453680b73bd2a5811d17: Sending DeleteTablet for 3 replicas of tablet d9a1500cb289435cb50c6985ee3fa2f2
I20260809 00:44:04.988776 15278 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a1500cb289435cb50c6985ee3fa2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:44:04 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:46310
I20260809 00:44:04.989071 14851 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a1500cb289435cb50c6985ee3fa2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:44:04 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:35976
I20260809 00:44:04.989653 15078 tablet_service.cc:1514] Processing DeleteTablet for tablet d9a1500cb289435cb50c6985ee3fa2f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:44:04 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:37384
I20260809 00:44:04.990127 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 14769
I20260809 00:44:04.999617 15429 tablet_replica.cc:330] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: stopping tablet replica
I20260809 00:44:04.999800 15429 raft_consensus.cc:2229] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:44:04.999930 15429 raft_consensus.cc:2258] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:44:05.000504 15429 ts_tablet_manager.cc:1905] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:44:05.002573 15429 ts_tablet_manager.cc:1918] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:44:05.002663 15429 log.cc:1196] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/d9a1500cb289435cb50c6985ee3fa2f2
I20260809 00:44:05.003414 15429 ts_tablet_manager.cc:1939] T d9a1500cb289435cb50c6985ee3fa2f2 P bcdb7bdb1eff4a8e83dc3b82017808bf: Deleting consensus metadata
I20260809 00:44:05.004004 14621 catalog_manager.cc:4841] TS bcdb7bdb1eff4a8e83dc3b82017808bf (127.30.104.195:43661): tablet d9a1500cb289435cb50c6985ee3fa2f2 (table test-table [id=07b7717f5946487080a08b46755af649]) successfully deleted
I20260809 00:44:05.007652 15433 tablet_replica.cc:330] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: stopping tablet replica
I20260809 00:44:05.007831 15433 raft_consensus.cc:2229] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:44:05.008021 15433 raft_consensus.cc:2258] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:44:05.012223 15433 ts_tablet_manager.cc:1905] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:44:05.014101 15433 ts_tablet_manager.cc:1918] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:44:05.014192 15433 log.cc:1196] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/d9a1500cb289435cb50c6985ee3fa2f2
I20260809 00:44:05.014477 15433 ts_tablet_manager.cc:1939] T d9a1500cb289435cb50c6985ee3fa2f2 P cc349c8fb211484282eda1063d12e18a: Deleting consensus metadata
I20260809 00:44:05.015268 14622 catalog_manager.cc:4841] TS cc349c8fb211484282eda1063d12e18a (127.30.104.194:34797): tablet d9a1500cb289435cb50c6985ee3fa2f2 (table test-table [id=07b7717f5946487080a08b46755af649]) successfully deleted
W20260809 00:44:05.016237 14626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:45431 (error 108)
I20260809 00:44:05.016966 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 14987
I20260809 00:44:05.031548 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:05.029942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45431 (local address 127.0.0.1:35986)
0809 00:44:05.031387 (+ 1445us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:05.031493 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 00:44:05.031911 14626 catalog_manager.cc:4584] TS 315061458efb4c0b90dd35644866f7ac (127.30.104.193:45431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9a1500cb289435cb50c6985ee3fa2f2: Network error: Client connection negotiation failed: client connection to 127.30.104.193:45431: connect: Connection refused (error 111)
I20260809 00:44:05.084710 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:05.084546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45431 (local address 127.0.0.1:36000)
0809 00:44:05.084625 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:05.084661 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 00:44:05.085187 15390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:05.085100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45431 (local address 127.0.0.1:36006)
0809 00:44:05.085138 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:05.085161 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 00:44:05.091717 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 15182
I20260809 00:44:05.108995 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 14592
I20260809 00:44:05.127002 31139 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 00:44:05.473358 31139 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 00:44:05.473500 31139 mini_ranger.cc:66] Stopping Ranger...
I20260809 00:44:05.818722 31139 mini_ranger.cc:68] Stopped Ranger
2026-08-09 00:44:05.818 UTC [13949] LOG: received smart shutdown request
2026-08-09 00:44:05.822 UTC [13949] LOG: background worker "logical replication launcher" (PID 13961) exited with exit code 1
2026-08-09 00:44:05.822 UTC [13955] LOG: shutting down
2026-08-09 00:44:06.011 UTC [13949] LOG: database system is shut down
2026-08-09T00:44:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52826 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:44:06 dist-test-slave-c12x krb5kdc[15450](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:44:06 dist-test-slave-c12x krb5kdc[15450](info): set up 2 sockets
Aug 09 00:44:06 dist-test-slave-c12x krb5kdc[15450](info): commencing operation
krb5kdc: starting...
W20260809 00:44:07.370604 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:44:07 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:44:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:44:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.104.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.104.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/rangeradmin_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/rangeradmin_127.30.104.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.104.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.104.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/rangerlookup_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/rangerlookup_127.30.104.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.104.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.104.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/HTTP_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/HTTP_127.30.104.212@KRBTEST.COM.keytab.
I20260809 00:44:07.559365 31139 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxSoRG4/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/postgres -l logfile start
2026-08-09 00:44:10.141 UTC [15501] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 00:44:10.142 UTC [15501] LOG: listening on IPv4 address "127.30.104.212", port 46769
2026-08-09 00:44:10.146 UTC [15501] LOG: listening on Unix socket "/tmp/.s.PGSQL.46769"
2026-08-09 00:44:10.186 UTC [15505] LOG: database system was shut down at 2026-08-09 00:44:08 UTC
2026-08-09 00:44:10.191 UTC [15501] LOG: database system is ready to accept connections
I20260809 00:44:11.346441 31139 mini_postgres.cc:96] Postgres bound to 46769
2026-08-09 00:44:11.353 UTC [15520] FATAL: database "slave" does not exist
127.30.104.212:46769 - accepting connections
I20260809 00:44:11.354310 31139 mini_ranger.cc:161] Starting Ranger...
I20260809 00:44:11.372632 31139 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 00:44:11.868046 31139 mini_ranger.cc:87] Created ranger Postgres database
I20260809 00:44:11.868148 31139 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-admin
2026-08-09 00:44:12,162 [I] DB FLAVOR :POSTGRES
2026-08-09 00:44:12,162 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:44:12,162 [I] Checking connection..
2026-08-09 00:44:12,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 00:44:12,441 [I] Checking connection passed.
2026-08-09 00:44:12,442 [I] --------- Verifying version history table ---------
2026-08-09 00:44:12,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:44:12,704 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:44:12,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:44:12,978 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 00:44:12,978 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 00:44:12,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 00:44:13.231 UTC [15611] WARNING: there is no transaction in progress
2026-08-09 00:44:13,238 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 00:44:13,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 00:44:13,494 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 00:44:13,494 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:44:13,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 00:44:13,776 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 00:44:14,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-c12x.c.gcp-upstream.internal','N') ;"
2026-08-09 00:44:14,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:44:14,566 [I] Table x_portal_user does not exist in database ranger
2026-08-09 00:44:14,566 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:44:14,834 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 00:44:14,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:44:15,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:44:15,347 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 00:44:15,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 00:44:16.395 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.414 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.433 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.453 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.472 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.495 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.561 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.568 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.584 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.595 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.606 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.622 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.633 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.640 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.648 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:16.656 UTC [15773] WARNING: there is no transaction in progress
2026-08-09 00:44:17,388 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 00:44:17,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 00:44:17,638 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 00:44:17,638 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 00:44:17,906 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 00:44:17,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 00:44:18,189 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 00:44:18,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 00:44:18,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 00:44:18,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-c12x.c.gcp-upstream.internal';"
2026-08-09 00:44:18,985 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 00:44:18,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 00:44:19,259 [I] DB_PATCHES have already been applied
I20260809 00:44:19.265447 31139 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxSoRG4/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 00:44:19.265524 31139 mini_ranger.cc:193] Using host: 127.30.104.212
I20260809 00:44:19.270011 31139 mini_ranger.cc:227] Ranger admin URL: http://127.30.104.212:53307
Aug 09, 2026 12:44:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:44:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:44:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:44:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:44:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:44:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:44:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 00:44:20.623819 31139 mini_ranger.cc:160] Time spent starting Ranger: real 9.270s user 0.000s sys 0.012s
Aug 09, 2026 12:44:42 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:44:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 12:44:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:44:42 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:44:43 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 00:44:43.731420 31139 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.104.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.104.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/rangerkms_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.104.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/rangerkms_127.30.104.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.104.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.104.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.104.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/HTTP_127.30.104.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.104.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/HTTP_127.30.104.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 00:44:43 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236283, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 00:44:43.846517 31139 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 00:44:43.859401 31139 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 00:44:44.786018 31139 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 00:44:44.786099 31139 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms
I20260809 00:44:44.786113 31139 mini_ranger_kms.cc:227] Using postgres at 127.30.104.212:46769
2026-08-09 00:44:44,966 [I] DB FLAVOR :POSTGRES
2026-08-09 00:44:44,966 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 00:44:44,967 [I] Checking connection
2026-08-09 00:44:44,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 00:44:45,221 [I] connection success
2026-08-09 00:44:45,222 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 00:44:45,222 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 00:44:45,222 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 00:44:45,474 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 00:44:45,475 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 00:44:45,475 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 00:44:45,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.104.212:46769/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 00:44:45,742 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 00:44:46.301563 31139 mini_ranger_kms.cc:310] Created kms service
I20260809 00:44:46.529240 31139 mini_ranger_kms.cc:326] Created kudu user
I20260809 00:44:46.573937 31139 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 00:44:46 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236283, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:47.036154 31139 mini_ranger_kms.cc:384] Added ranger policy
I20260809 00:44:47.036291 31139 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxSoRG4/build/release/bin/postgresql.jar:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxSoRG4/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 00:44:47.036314 31139 mini_ranger_kms.cc:240] Using host: 127.30.104.212
I20260809 00:44:47.041312 31139 mini_ranger_kms.cc:276] Ranger KMS PID: 16060
I20260809 00:44:47.041412 31139 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.104.212:55491
Aug 09, 2026 12:44:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:44:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:44:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:44:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:44:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:44:48 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 00:44:49 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786236289, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.104.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:49.765830 31139 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.919s user 0.006s sys 0.160s
I20260809 00:44:49.765944 31139 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 00:44:49.765995 31139 mini_ranger_kms.cc:401] 127.30.104.212:55491/kms/v1/keys
Aug 09 00:44:49 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236289, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.104.212@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
Aug 09 00:44:50 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236289, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.104.212@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
Aug 09 00:44:52 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:44:52.894745 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:41677
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:41677
--ranger_config_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:55491/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:52.982999 16134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:52.983232 16134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:52.983256 16134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:52.984557 16134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:44:52.984612 16134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:52.984627 16134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:52.984639 16134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:44:52.984651 16134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:44:52.986351 16134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.104.212:55491/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:41677
--ranger_config_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:41677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:52.986634 16134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:52.986913 16134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:52.992494 16145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:52.992434 16147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:52.992434 16142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:52.993155 16134 server_base.cc:1034] running on GCE node
I20260809 00:44:52.993417 16134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:52.993768 16134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:52.994932 16134 hybrid_clock.cc:648] HybridClock initialized: now 1786236292994905 us; error 45 us; skew 500 ppm
Aug 09 00:44:52 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:52.997599 16134 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:44:52.998135 16134 webserver.cc:460] Webserver started at http://127.30.104.254:36423/ using document root <none> and password file <none>
I20260809 00:44:52.998378 16134 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:52.998420 16134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:52.998558 16134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:44:52 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236292, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:53.062564 16134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "793b1a3a012741a0accfb0d5f940a245"
format_stamp: "Formatted at 2026-08-09 00:44:53 on dist-test-slave-c12x"
server_key: "749b51433b6ca10838b9f0163580e394"
server_key_iv: "d54a3885a3bc6bfb7658b1dad42932a1"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:53.063097 16134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "793b1a3a012741a0accfb0d5f940a245"
format_stamp: "Formatted at 2026-08-09 00:44:53 on dist-test-slave-c12x"
server_key: "749b51433b6ca10838b9f0163580e394"
server_key_iv: "d54a3885a3bc6bfb7658b1dad42932a1"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:53.064998 16134 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:44:53.080365 16161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:53.080695 16134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:44:53.080816 16134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "793b1a3a012741a0accfb0d5f940a245"
format_stamp: "Formatted at 2026-08-09 00:44:53 on dist-test-slave-c12x"
server_key: "749b51433b6ca10838b9f0163580e394"
server_key_iv: "d54a3885a3bc6bfb7658b1dad42932a1"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:53.080926 16134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:53.094769 16134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:53.097079 16134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:53.097239 16134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:53.102623 16134 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:41677
I20260809 00:44:53.102721 16226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:41677 every 8 connection(s)
I20260809 00:44:53.103144 16134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:44:53.104041 16227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:53.106482 16227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Bootstrap starting.
I20260809 00:44:53.107139 16227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:53.107504 16227 log.cc:824] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:53.108317 16227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: No bootstrap required, opened a new log
I20260809 00:44:53.110450 16227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } }
I20260809 00:44:53.110632 16227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:53.110689 16227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 793b1a3a012741a0accfb0d5f940a245, State: Initialized, Role: FOLLOWER
I20260809 00:44:53.110867 16227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } }
I20260809 00:44:53.110975 16227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:44:53.111021 16227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:44:53.111081 16227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:53.111749 16227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } }
I20260809 00:44:53.111879 16227 leader_election.cc:302] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 793b1a3a012741a0accfb0d5f940a245; no voters:
I20260809 00:44:53.112097 16227 leader_election.cc:288] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:44:53.112157 16233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:44:53.112329 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 16134
I20260809 00:44:53.112432 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:44:53.112457 16233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 1 LEADER]: Becoming Leader. State: Replica: 793b1a3a012741a0accfb0d5f940a245, State: Running, Role: LEADER
I20260809 00:44:53.112639 31139 external_mini_cluster.cc:1428] Setting key 5eb17b6911468b221293da3c1faac9be
I20260809 00:44:53.112746 16233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } }
I20260809 00:44:53.112473 16227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:44:53.113201 16234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "793b1a3a012741a0accfb0d5f940a245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } } }
I20260809 00:44:53.113286 16234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:53.113533 16235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 793b1a3a012741a0accfb0d5f940a245. Latest consensus state: current_term: 1 leader_uuid: "793b1a3a012741a0accfb0d5f940a245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } } }
I20260809 00:44:53.113656 16235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:53.113829 16242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:44:53.114429 16242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:44:53.114738 16227 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 00:44:53.117427 16268 mvcc.cc:205] Tried to move back new op lower bound from 7316423856601227264 to 7316423856590135296. Current Snapshot: MvccSnapshot[applied={T|T < 7316423856601227264}]
I20260809 00:44:53.117945 16242 catalog_manager.cc:1347] Generated new cluster ID: 350f7b3f97de47d5a6a083518cfe32b5
I20260809 00:44:53.118000 16242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:44:53 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236292, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:53.127199 16241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:53.113287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.30.104.254:41677)
0809 00:44:53.113706 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:53.113717 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:53.113754 (+ 37us) server_negotiation.cc:407] Connection header received
0809 00:44:53.113887 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:53.113896 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:53.113957 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:53.114094 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:53.115052 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:53.116501 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:53.117666 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:53.118323 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:53.118432 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:53.122568 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:53.122589 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:53.122589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:53.122615 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:53.124930 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:53.125652 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:53.125656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:53.125656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:53.125773 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:53.126001 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:53.126002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:53.126002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:53.126379 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:53.126501 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:53.126771 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 00:44:53.126938 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":71,"threads_started":1}
I20260809 00:44:53.154335 16242 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:44:53.155112 16242 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:44:53.165031 16242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Generated new TSK 0
I20260809 00:44:53.165334 16242 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:44:53.843163 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:55491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41677
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:53.959074 16305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:53.959362 16305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:53.959406 16305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:53.960943 16305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:53.961068 16305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:53.961133 16305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:44:53.963320 16305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.104.212:55491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:53.963766 16305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:53.964087 16305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:53.970584 16317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:53.970937 16316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:53.970966 16319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:53.971899 16305 server_base.cc:1034] running on GCE node
I20260809 00:44:53.972165 16305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:53.972455 16305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:53.973704 16305 hybrid_clock.cc:648] HybridClock initialized: now 1786236293973683 us; error 38 us; skew 500 ppm
Aug 09 00:44:53 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:53.976805 16305 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:44:53.977362 16305 webserver.cc:460] Webserver started at http://127.30.104.193:44065/ using document root <none> and password file <none>
I20260809 00:44:53.977694 16305 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:53.977779 16305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:53.977924 16305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:44:53 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:53.992839 16305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "365a500341344d59b1bb5eb093e95fd6"
format_stamp: "Formatted at 2026-08-09 00:44:53 on dist-test-slave-c12x"
server_key: "76fb5b810db28c4ecc3f79bc87e6cffa"
server_key_iv: "57f57f844616bbe81d580d5f7b34dac7"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:53.993336 16305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "365a500341344d59b1bb5eb093e95fd6"
format_stamp: "Formatted at 2026-08-09 00:44:53 on dist-test-slave-c12x"
server_key: "76fb5b810db28c4ecc3f79bc87e6cffa"
server_key_iv: "57f57f844616bbe81d580d5f7b34dac7"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:53.994946 16305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:44:54.011617 16330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.011865 16305 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 00:44:54.011952 16305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "365a500341344d59b1bb5eb093e95fd6"
format_stamp: "Formatted at 2026-08-09 00:44:53 on dist-test-slave-c12x"
server_key: "76fb5b810db28c4ecc3f79bc87e6cffa"
server_key_iv: "57f57f844616bbe81d580d5f7b34dac7"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:54.012050 16305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:54.030333 16305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:54.032979 16305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:54.033209 16305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:54.033506 16305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:54.033921 16305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:54.034022 16305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.034118 16305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:54.034193 16305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.041632 16305 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:46407
I20260809 00:44:54.042160 16305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:44:54.044716 16516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:46407 every 8 connection(s)
I20260809 00:44:54.051296 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 16305
I20260809 00:44:54.051414 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:44:54.051654 31139 external_mini_cluster.cc:1428] Setting key 5cd171ab2798a664e6155396adcce5d0
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236293, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:54.073091 16521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:54.044743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41677 (local address 127.30.104.193:48001)
0809 00:44:54.052009 (+ 7266us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:54.052085 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:54.055882 (+ 3797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:54.062680 (+ 6798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:54.062687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:54.063000 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:54.063333 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:54.063341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.064578 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.064579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:54.065916 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:54.065919 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.066464 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.066466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:54.066610 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:54.067268 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:54.067280 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:54.069249 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:54.071392 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:54.071400 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:54.071402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:54.071746 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:54.071966 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:54.071966 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:54.071967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:54.072097 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:54.072384 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:54.072389 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:54.072571 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 00:44:54.072672 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 00:44:54.072839 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7200,"thread_start_us":87,"threads_started":1}
I20260809 00:44:54.073908 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:54.044749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:48001 (local address 127.30.104.254:41677)
0809 00:44:54.062340 (+ 17591us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:54.062343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:54.062361 (+ 18us) server_negotiation.cc:407] Connection header received
0809 00:44:54.062396 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:54.062397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:54.062412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:54.063569 (+ 1157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:54.063616 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.064455 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.066036 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.066356 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.066894 (+ 538us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:54.069372 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:54.069374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:54.069374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:54.069384 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:54.071294 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:54.071834 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:54.071835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:54.071836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:54.071901 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:54.072164 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:54.072164 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:54.072165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:54.072302 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:54.072359 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:54.073713 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0809 00:44:54.073805 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17553,"thread_start_us":87,"threads_started":1}
I20260809 00:44:54.074499 16517 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41677
I20260809 00:44:54.074600 16517 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:54.074793 16517 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:54.075337 16183 ts_manager.cc:194] Registered new tserver with Master: 365a500341344d59b1bb5eb093e95fd6 (127.30.104.193:46407)
I20260809 00:44:54.076370 16183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:48001
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:44:54.100631 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:55491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41677
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 00:44:54.197814 16531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:54.198167 16531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:54.198252 16531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:54.200100 16531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:54.200223 16531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:54.200313 16531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:44:54.202412 16531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.104.212:55491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:54.202878 16531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:54.203235 16531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:54.208276 16540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:54.208276 16541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:54.208701 16543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:54.209530 16531 server_base.cc:1034] running on GCE node
I20260809 00:44:54.209755 16531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:54.210031 16531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:54.211254 16531 hybrid_clock.cc:648] HybridClock initialized: now 1786236294211229 us; error 41 us; skew 500 ppm
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:54.214140 16531 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:44:54.214571 16531 webserver.cc:460] Webserver started at http://127.30.104.194:46443/ using document root <none> and password file <none>
I20260809 00:44:54.214838 16531 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:54.214910 16531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:54.215126 16531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:54.233430 16531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "4643cadb28c444eba95664893d067613"
format_stamp: "Formatted at 2026-08-09 00:44:54 on dist-test-slave-c12x"
server_key: "86ec878204617d970219d9aeef320e32"
server_key_iv: "66aff143917d487bd07d5bcef5aa0c3b"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:54.233920 16531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "4643cadb28c444eba95664893d067613"
format_stamp: "Formatted at 2026-08-09 00:44:54 on dist-test-slave-c12x"
server_key: "86ec878204617d970219d9aeef320e32"
server_key_iv: "66aff143917d487bd07d5bcef5aa0c3b"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:54.235692 16531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:44:54.255837 16555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.256131 16531 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.000s
I20260809 00:44:54.256247 16531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "4643cadb28c444eba95664893d067613"
format_stamp: "Formatted at 2026-08-09 00:44:54 on dist-test-slave-c12x"
server_key: "86ec878204617d970219d9aeef320e32"
server_key_iv: "66aff143917d487bd07d5bcef5aa0c3b"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:54.256357 16531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:54.276399 16531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:54.279776 16531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:54.280020 16531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:54.280309 16531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:54.280918 16531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:54.281015 16531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.281098 16531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:54.281160 16531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.287993 16531 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:34379
I20260809 00:44:54.288542 16531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:44:54.289243 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 16531
I20260809 00:44:54.289350 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:44:54.289556 31139 external_mini_cluster.cc:1428] Setting key acc6ada82e4b57bd2833f384c5182418
I20260809 00:44:54.299947 16736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:34379 every 8 connection(s)
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
I20260809 00:44:54.311861 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:54.290280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:59113 (local address 127.30.104.254:41677)
0809 00:44:54.290380 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:54.290382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:54.296459 (+ 6077us) server_negotiation.cc:407] Connection header received
0809 00:44:54.297419 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:54.297421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:54.297443 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:54.297480 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:54.298478 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.299290 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.300922 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.301273 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.301332 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:54.308043 (+ 6711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:54.308046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:54.308046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:54.308056 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:54.310005 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:54.310615 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:54.310616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:54.310617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:54.310680 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:54.310971 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:54.310972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:54.310972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:54.311138 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:54.311191 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:54.311652 (+ 461us) server_negotiation.cc:299] Negotiation successful
0809 00:44:54.311760 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 00:44:54.312745 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:54.290012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41677 (local address 127.30.104.194:59113)
0809 00:44:54.296259 (+ 6247us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:54.296331 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:54.297254 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:54.297714 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:54.297720 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:54.298039 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:54.298350 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:54.298357 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.299422 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.299422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:54.300815 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:54.300818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.301457 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.301458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:54.301573 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:54.302263 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:54.302274 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:54.307870 (+ 5596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:54.310130 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:54.310140 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:54.310143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:54.310504 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:54.310746 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:54.310747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:54.310747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:54.310891 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:54.311226 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:54.311233 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:54.311459 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 00:44:54.312286 (+ 827us) client_negotiation.cc:240] Negotiation successful
0809 00:44:54.312482 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6186,"thread_start_us":92,"threads_started":1}
I20260809 00:44:54.313836 16737 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41677
I20260809 00:44:54.313951 16737 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:54.314220 16737 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:54.314688 16183 ts_manager.cc:194] Registered new tserver with Master: 4643cadb28c444eba95664893d067613 (127.30.104.194:34379)
I20260809 00:44:54.315209 16183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:59113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:44:54.331825 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:55491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41677
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:54.449731 16743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:54.450127 16743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:54.450220 16743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:54.452159 16743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:54.452302 16743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:54.452397 16743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:44:54.455049 16743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.104.212:55491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:54.455597 16743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:54.455948 16743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:54.463198 16748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:54.463657 16753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:54.463946 16751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:54.464990 16743 server_base.cc:1034] running on GCE node
I20260809 00:44:54.465415 16743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:54.465731 16743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:54.466912 16743 hybrid_clock.cc:648] HybridClock initialized: now 1786236294466889 us; error 38 us; skew 500 ppm
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:54.469970 16743 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:44:54.470435 16743 webserver.cc:460] Webserver started at http://127.30.104.195:41221/ using document root <none> and password file <none>
I20260809 00:44:54.470753 16743 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:54.470832 16743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:54.470990 16743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:54.483630 16743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "12da735cd6604daca3c1a8066868ef8c"
format_stamp: "Formatted at 2026-08-09 00:44:54 on dist-test-slave-c12x"
server_key: "f20269b771418be693e55b0b6e61638c"
server_key_iv: "42ba7a4a3ff289cc46c7c8304cd70965"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:54.484229 16743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "12da735cd6604daca3c1a8066868ef8c"
format_stamp: "Formatted at 2026-08-09 00:44:54 on dist-test-slave-c12x"
server_key: "f20269b771418be693e55b0b6e61638c"
server_key_iv: "42ba7a4a3ff289cc46c7c8304cd70965"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:54.491320 16743 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260809 00:44:54.508539 16761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.508819 16743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 00:44:54.508914 16743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "12da735cd6604daca3c1a8066868ef8c"
format_stamp: "Formatted at 2026-08-09 00:44:54 on dist-test-slave-c12x"
server_key: "f20269b771418be693e55b0b6e61638c"
server_key_iv: "42ba7a4a3ff289cc46c7c8304cd70965"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:54.509014 16743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:54.542771 16743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:54.546486 16743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:54.546720 16743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:54.546988 16743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:54.547416 16743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:54.547537 16743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.547776 16743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:54.547839 16743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.563275 16743 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:33457
I20260809 00:44:54.563900 16743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:44:54.570112 16946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:33457 every 8 connection(s)
I20260809 00:44:54.572422 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 16743
I20260809 00:44:54.572530 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:44:54.572736 31139 external_mini_cluster.cc:1428] Setting key d828439d5b6ba1ccb9cf7121444b49a6
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:54.591619 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:54.570830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:53477 (local address 127.30.104.254:41677)
0809 00:44:54.570896 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:54.570899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:54.575744 (+ 4845us) server_negotiation.cc:407] Connection header received
0809 00:44:54.576458 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:54.576459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:54.576472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:54.576504 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:54.577383 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.578147 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.581879 (+ 3732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.582197 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.582245 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:54.585380 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:54.585382 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:54.585382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:54.585390 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:54.589427 (+ 4037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:54.590484 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:54.590485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:54.590486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:54.590547 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:54.590833 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:54.590834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:54.590834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:54.590970 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:54.591020 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:54.591400 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 00:44:54.591521 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:44:54.592391 16950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:54.570845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41677 (local address 127.30.104.195:53477)
0809 00:44:54.575590 (+ 4745us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:54.575651 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:54.576333 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:54.576701 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:54.576705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:54.576967 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:54.577269 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:54.577276 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.578270 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.578270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:54.581768 (+ 3498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:54.581772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:54.582364 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:54.582365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:54.582513 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:54.583195 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:54.583206 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:54.585266 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:54.590069 (+ 4803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:54.590077 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:54.590080 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:54.590389 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:54.590616 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:54.590617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:54.590617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:54.590762 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:54.591063 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:54.591070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:54.591236 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 00:44:54.591961 (+ 725us) client_negotiation.cc:240] Negotiation successful
0809 00:44:54.592152 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4691,"thread_start_us":100,"threads_started":1}
I20260809 00:44:54.593087 16948 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41677
I20260809 00:44:54.593182 16948 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:54.593366 16948 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:54.593775 16183 ts_manager.cc:194] Registered new tserver with Master: 12da735cd6604daca3c1a8066868ef8c (127.30.104.195:33457)
I20260809 00:44:54.594252 16183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:53477
I20260809 00:44:54.595292 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:44:54.595971 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 16305
I20260809 00:44:54.633872 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 16531
I20260809 00:44:54.644201 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 16743
I20260809 00:44:54.659570 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 16134
I20260809 00:44:54.674144 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:41677
--webserver_interface=127.30.104.254
--webserver_port=36423
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:41677
--ranger_config_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:55491/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.104.212:55491/kms with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:54.759027 16951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:54.759249 16951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:54.759272 16951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:54.760470 16951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:44:54.760517 16951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:54.760540 16951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:54.760552 16951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:44:54.760564 16951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:44:54.762217 16951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.104.212:55491/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:41677
--ranger_config_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:41677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=36423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:54.762480 16951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:54.762733 16951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:54.768635 16960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:54.768642 16962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:54.768744 16966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:54.768801 16951 server_base.cc:1034] running on GCE node
I20260809 00:44:54.769111 16951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:54.769464 16951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:54.770640 16951 hybrid_clock.cc:648] HybridClock initialized: now 1786236294770617 us; error 37 us; skew 500 ppm
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:54.773416 16951 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:44:54.773937 16951 webserver.cc:460] Webserver started at http://127.30.104.254:36423/ using document root <none> and password file <none>
I20260809 00:44:54.774138 16951 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:54.774194 16951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:44:54 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236294, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:54.853223 16951 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:44:54.854203 16978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:54.854426 16951 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:44:54.854540 16951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "793b1a3a012741a0accfb0d5f940a245"
format_stamp: "Formatted at 2026-08-09 00:44:53 on dist-test-slave-c12x"
server_key: "749b51433b6ca10838b9f0163580e394"
server_key_iv: "d54a3885a3bc6bfb7658b1dad42932a1"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:54.855038 16951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:54.896864 16951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:54.898646 16951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:54.898774 16951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:54.903712 16951 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:41677
I20260809 00:44:54.903769 17052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:41677 every 8 connection(s)
I20260809 00:44:54.904179 16951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:44:54.905539 17054 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:44:54.906219 17054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Bootstrap starting.
I20260809 00:44:54.912551 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 16951
I20260809 00:44:54.912879 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:55491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:46407
--local_ip_for_outbound_sockets=127.30.104.193
--tserver_master_addrs=127.30.104.254:41677
--webserver_port=44065
--webserver_interface=127.30.104.193
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.104.212:55491/kms with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:44:54.915078 17054 log.cc:824] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:54.917058 17054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:44:54.917397 17054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Bootstrap complete.
I20260809 00:44:54.920168 17054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } }
I20260809 00:44:54.920450 17054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 793b1a3a012741a0accfb0d5f940a245, State: Initialized, Role: FOLLOWER
I20260809 00:44:54.920658 17054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } }
I20260809 00:44:54.920748 17054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:44:54.920807 17054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:44:54.920861 17054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:44:54.921940 17054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } }
I20260809 00:44:54.922081 17054 leader_election.cc:302] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 793b1a3a012741a0accfb0d5f940a245; no voters:
I20260809 00:44:54.922313 17054 leader_election.cc:288] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:44:54.922353 17058 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:44:54.922483 17058 raft_consensus.cc:686] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [term 2 LEADER]: Becoming Leader. State: Replica: 793b1a3a012741a0accfb0d5f940a245, State: Running, Role: LEADER
I20260809 00:44:54.922634 17054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:44:54.922665 17058 consensus_queue.cc:237] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } }
I20260809 00:44:54.923061 17059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "793b1a3a012741a0accfb0d5f940a245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } } }
I20260809 00:44:54.923146 17059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:54.923524 17061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 793b1a3a012741a0accfb0d5f940a245. Latest consensus state: current_term: 2 leader_uuid: "793b1a3a012741a0accfb0d5f940a245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "793b1a3a012741a0accfb0d5f940a245" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 41677 } } }
I20260809 00:44:54.923619 17061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:54.923717 17067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:44:54.924424 17067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:44:54.924583 17067 catalog_manager.cc:1259] Loaded cluster ID: 350f7b3f97de47d5a6a083518cfe32b5
I20260809 00:44:54.924680 17067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:44:54.924919 17054 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 00:44:54.928768 17067 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:44:54.929291 17067 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Loaded TSK: 0
I20260809 00:44:54.929626 17067 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:44:55.004640 17056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:55.004869 17056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:55.004910 17056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:55.006192 17056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:55.006255 17056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:55.006300 17056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:44:55.008538 17056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.104.212:55491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:46407
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=44065
--tserver_master_addrs=127.30.104.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:55.009043 17056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:55.009423 17056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:55.016780 17108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:55.016841 17106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:55.016798 17105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:55.017234 17056 server_base.cc:1034] running on GCE node
I20260809 00:44:55.017505 17056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:55.017854 17056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:55.019040 17056 hybrid_clock.cc:648] HybridClock initialized: now 1786236295019002 us; error 51 us; skew 500 ppm
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:55.022576 17056 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:44:55.023211 17056 webserver.cc:460] Webserver started at http://127.30.104.193:44065/ using document root <none> and password file <none>
I20260809 00:44:55.023629 17056 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:55.023726 17056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:55.056922 17056 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260809 00:44:55.058100 17123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.058326 17056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:44:55.058422 17056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "365a500341344d59b1bb5eb093e95fd6"
format_stamp: "Formatted at 2026-08-09 00:44:53 on dist-test-slave-c12x"
server_key: "76fb5b810db28c4ecc3f79bc87e6cffa"
server_key_iv: "57f57f844616bbe81d580d5f7b34dac7"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:55.058898 17056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:55.082502 17056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:55.085322 17056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:55.085518 17056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:55.085745 17056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:55.086161 17056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:55.086206 17056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.086236 17056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:55.086285 17056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.099303 17056 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:46407
I20260809 00:44:55.099913 17056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:44:55.100343 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 17056
I20260809 00:44:55.100625 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:55491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:34379
--local_ip_for_outbound_sockets=127.30.104.194
--tserver_master_addrs=127.30.104.254:41677
--webserver_port=46443
--webserver_interface=127.30.104.194
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.104.212:55491/kms with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:44:55.109443 17283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:46407 every 8 connection(s)
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:55.128571 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:55.111695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41677 (local address 127.30.104.193:56643)
0809 00:44:55.112562 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:55.112619 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:55.113323 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:55.114203 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:55.114208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:55.114484 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:55.114785 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:55.114792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.117140 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.117141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:55.118978 (+ 1837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:55.118981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.120543 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.120544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:55.120690 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:55.121369 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:55.121388 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:55.123179 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:55.126489 (+ 3310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:55.126496 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:55.126499 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:55.126903 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:55.127207 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:55.127208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:55.127209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:55.127336 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:55.127858 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:55.127866 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:55.128031 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 00:44:55.128141 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 00:44:55.128307 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":112,"threads_started":1}
I20260809 00:44:55.129894 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:55.111732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:56643 (local address 127.30.104.254:41677)
0809 00:44:55.113577 (+ 1845us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:55.113588 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:55.113631 (+ 43us) server_negotiation.cc:407] Connection header received
0809 00:44:55.113791 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:55.113798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:55.113858 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:55.114029 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:55.114939 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.116978 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.119327 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.120309 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.120420 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:55.123324 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:55.123328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:55.123329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:55.123342 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:55.126384 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:55.127012 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:55.127016 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:55.127016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:55.127130 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:55.127409 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:55.127409 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:55.127410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:55.127739 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:55.127813 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:55.129433 (+ 1620us) server_negotiation.cc:299] Negotiation successful
0809 00:44:55.129596 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1782,"thread_start_us":78,"threads_started":1}
I20260809 00:44:55.130926 17286 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41677
I20260809 00:44:55.131039 17286 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.131237 17286 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
W20260809 00:44:55.132864 17286 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:41677 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:44:55.132961 17286 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.133062 17286 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:55.133591 17286 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.133687 17286 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
W20260809 00:44:55.134073 17286 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:44:55.227778 17284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:55.228091 17284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:55.228142 17284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:55.229952 17284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:55.230024 17284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:55.230070 17284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:44:55.232542 17284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.104.212:55491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:34379
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=46443
--tserver_master_addrs=127.30.104.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:55.232822 17284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:55.233141 17284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:44:55.239748 17284 server_base.cc:1034] running on GCE node
W20260809 00:44:55.239852 17300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:55.240458 17297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:55.240491 17298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:55.240724 17284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:55.241030 17284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:55.242182 17284 hybrid_clock.cc:648] HybridClock initialized: now 1786236295242170 us; error 27 us; skew 500 ppm
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:55.245221 17284 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:44:55.245787 17284 webserver.cc:460] Webserver started at http://127.30.104.194:46443/ using document root <none> and password file <none>
I20260809 00:44:55.246078 17284 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:55.246208 17284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:55.263360 17284 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:44:55.265079 17314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.265333 17284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:44:55.265462 17284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "4643cadb28c444eba95664893d067613"
format_stamp: "Formatted at 2026-08-09 00:44:54 on dist-test-slave-c12x"
server_key: "86ec878204617d970219d9aeef320e32"
server_key_iv: "66aff143917d487bd07d5bcef5aa0c3b"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:55.265995 17284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:55.281267 17284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:55.283972 17284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:55.284191 17284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:55.284479 17284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:55.284929 17284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:55.284992 17284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.285029 17284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:55.285077 17284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.293112 17284 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:34379
I20260809 00:44:55.293219 17476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:34379 every 8 connection(s)
I20260809 00:44:55.293642 17284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:55.301126 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 17284
I20260809 00:44:55.301451 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.104.212:55491/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:33457
--local_ip_for_outbound_sockets=127.30.104.195
--tserver_master_addrs=127.30.104.254:41677
--webserver_port=41221
--webserver_interface=127.30.104.195
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.104.212:55491/kms with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:44:55.305017 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:55.294547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:47849 (local address 127.30.104.254:41677)
0809 00:44:55.294657 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:55.294660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:55.294963 (+ 303us) server_negotiation.cc:407] Connection header received
0809 00:44:55.295517 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:55.295519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:55.295532 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:55.295616 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:55.296205 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.296989 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.298101 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.298335 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.298372 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:55.301373 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:55.301375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:55.301375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:55.301383 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:55.303213 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:55.303808 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:55.303809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:55.303809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:55.303876 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:55.304184 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:55.304185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:55.304185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:55.304338 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:55.304388 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:55.304822 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 00:44:55.304922 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 00:44:55.305819 17480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:55.294586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41677 (local address 127.30.104.194:47849)
0809 00:44:55.294787 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:55.294836 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:55.295353 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:55.295678 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:55.295682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:55.295881 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:55.296097 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:55.296101 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.297093 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.297094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:55.297954 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:55.297956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.298473 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.298473 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:55.298585 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:55.299227 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:55.299238 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:55.301243 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:55.303319 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:55.303327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:55.303330 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:55.303705 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:55.303957 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:55.303957 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:55.303958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:55.304097 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:55.304457 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:55.304463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:55.304645 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 00:44:55.305374 (+ 729us) client_negotiation.cc:240] Negotiation successful
0809 00:44:55.305573 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
I20260809 00:44:55.306640 17477 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41677
I20260809 00:44:55.306751 17477 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.306962 17477 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
W20260809 00:44:55.307791 17477 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:41677 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:44:55.307883 17477 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.307965 17477 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:55.308373 17477 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.308456 17477 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
W20260809 00:44:55.308867 17477 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:44:55.420965 17481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:55.421344 17481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:55.421437 17481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:55.422952 17481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:55.423031 17481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:55.423080 17481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:44:55.425555 17481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.104.212:55491/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:33457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=41221
--tserver_master_addrs=127.30.104.254:41677
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:55.425976 17481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:55.426309 17481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:55.433461 17494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:55.433477 17492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:55.435622 17496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:55.438267 17481 server_base.cc:1034] running on GCE node
I20260809 00:44:55.438570 17481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:55.438966 17481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:55.440223 17481 hybrid_clock.cc:648] HybridClock initialized: now 1786236295440209 us; error 46 us; skew 500 ppm
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:55.443715 17481 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:44:55.444314 17481 webserver.cc:460] Webserver started at http://127.30.104.195:41221/ using document root <none> and password file <none>
I20260809 00:44:55.444590 17481 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:55.444670 17481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:55.468657 17481 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 00:44:55.478524 17509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.478801 17481 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.000s
I20260809 00:44:55.478907 17481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "12da735cd6604daca3c1a8066868ef8c"
format_stamp: "Formatted at 2026-08-09 00:44:54 on dist-test-slave-c12x"
server_key: "f20269b771418be693e55b0b6e61638c"
server_key_iv: "42ba7a4a3ff289cc46c7c8304cd70965"
server_key_version: "kuduclusterkey@0"
I20260809 00:44:55.480314 17481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:55.494177 17481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:55.496740 17481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:55.496976 17481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:55.497259 17481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:44:55.497696 17481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:44:55.497786 17481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.497862 17481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:44:55.498008 17481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:55.506372 17481 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:33457
I20260809 00:44:55.506871 17481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:44:55.507602 17664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:33457 every 8 connection(s)
Aug 09 00:44:55 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236295, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:55.516520 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 17481
I20260809 00:44:55.519817 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:55.508393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:41677 (local address 127.30.104.195:45443)
0809 00:44:55.508776 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:55.508822 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:55.509483 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:55.509797 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:55.509801 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:55.510058 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:55.510343 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:55.510349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.511383 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.511384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:55.512639 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:55.512643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.513190 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.513191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:55.513328 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:55.513983 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:55.513994 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:55.515614 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:55.517925 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:55.517933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:55.517936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:55.518279 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:55.518528 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:55.518529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:55.518530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:55.518683 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:55.519016 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:55.519023 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:55.519206 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 00:44:55.519329 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 00:44:55.519560 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":93,"threads_started":1}
I20260809 00:44:55.520183 17291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:55.508691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:45443 (local address 127.30.104.254:41677)
0809 00:44:55.508787 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:55.508789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:55.508917 (+ 128us) server_negotiation.cc:407] Connection header received
0809 00:44:55.509611 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:55.509613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:55.509629 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:55.509717 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:55.510472 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.511252 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.512748 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.513028 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.513073 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:55.515731 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:55.515733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:55.515733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:55.515742 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:55.517821 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:55.518374 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:55.518376 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:55.518376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:55.518455 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:55.518769 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:55.518770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:55.518770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:55.518915 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:55.518983 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:55.519995 (+ 1012us) server_negotiation.cc:299] Negotiation successful
0809 00:44:55.520085 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:44:55.520743 17665 heartbeater.cc:347] Connected to a master server at 127.30.104.254:41677
I20260809 00:44:55.520854 17665 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.521059 17665 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
W20260809 00:44:55.521821 17665 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:41677 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 00:44:55.521898 17665 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.521963 17665 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:55.522481 17665 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:55.522574 17665 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
W20260809 00:44:55.522959 17665 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:44:55.527000 17669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:55.516979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.30.104.254:41677)
0809 00:44:55.517110 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:55.517112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:55.517326 (+ 214us) server_negotiation.cc:407] Connection header received
0809 00:44:55.517349 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:55.517350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:55.517375 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:55.517435 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:55.518196 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.518986 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.520902 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:55.521191 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:55.521428 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:55.523657 (+ 2229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:55.523659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:55.523659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:55.523666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:55.525437 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:55.525828 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:55.525830 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:55.525830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:55.525894 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:55.526093 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:55.526093 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:55.526094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:55.526222 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:55.526396 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:55.526809 (+ 413us) server_negotiation.cc:299] Negotiation successful
0809 00:44:55.526896 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260809 00:44:56.134269 17286 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:56.134445 17286 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:56.134918 17000 ts_manager.cc:194] Registered new tserver with Master: 365a500341344d59b1bb5eb093e95fd6 (127.30.104.193:46407)
I20260809 00:44:56.135960 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:56643
Aug 09 00:44:56 dist-test-slave-c12x krb5kdc[15450](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786236296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:56.309079 17477 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:56.309280 17477 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:56.309756 17000 ts_manager.cc:194] Registered new tserver with Master: 4643cadb28c444eba95664893d067613 (127.30.104.194:34379)
I20260809 00:44:56.310310 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:47849
Aug 09 00:44:56 dist-test-slave-c12x krb5kdc[15450](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236296, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for HTTP/127.30.104.212@KRBTEST.COM
I20260809 00:44:56.523133 17665 heartbeater.cc:464] Registering TS with master...
I20260809 00:44:56.523308 17665 heartbeater.cc:511] Master 127.30.104.254:41677 requested a full tablet report, sending...
I20260809 00:44:56.523665 17000 ts_manager.cc:194] Registered new tserver with Master: 12da735cd6604daca3c1a8066868ef8c (127.30.104.195:33457)
I20260809 00:44:56.524199 17000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:45443
I20260809 00:44:56.533118 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:44:56.543327 17697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.534220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48306 (local address 127.30.104.254:41677)
0809 00:44:56.534498 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:56.534502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:56.534671 (+ 169us) server_negotiation.cc:407] Connection header received
0809 00:44:56.534694 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:56.534695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:56.534707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:56.534763 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:56.535564 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.536386 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.537482 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.537803 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.537869 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.539480 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:56.539481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:56.539481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:56.539489 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:56.541549 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:56.542139 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:56.542140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:56.542141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:56.542211 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:56.542407 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:56.542407 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:56.542408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:56.542563 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:56.542656 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:56.543116 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 00:44:56.543208 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":78,"threads_started":1}
I20260809 00:44:56.546308 17000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:44:56.548642 17000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:44:56.564551 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.552769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:33457 (local address 127.0.0.1:49380)
0809 00:44:56.559591 (+ 6822us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:56.559648 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:56.559872 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:56.560249 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:56.560253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:56.560255 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:56.560337 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.560339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.561632 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.561633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.562957 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.562960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.564201 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.564202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.564317 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.564332 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:44:56.564397 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:44:56.564451 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6770,"thread_start_us":92,"threads_started":1}
I20260809 00:44:56.564812 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.552933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49380 (local address 127.30.104.195:33457)
0809 00:44:56.553253 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:56.553288 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:56.559731 (+ 6443us) server_negotiation.cc:407] Connection header received
0809 00:44:56.560012 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:56.560016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:56.560035 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:56.560129 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:56.560435 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.561522 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.563079 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.563933 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.564013 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.564138 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:56.564684 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 00:44:56.564732 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":121,"threads_started":1}
I20260809 00:44:56.565653 17562 tablet_service.cc:1467] Processing CreateTablet for tablet 28e74925b6484113b9a9e6c42551e3ef (DEFAULT_TABLE table=test-table [id=11e438bbe9a54f269317ba9c6fc1eded]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:44:56.566068 17562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e74925b6484113b9a9e6c42551e3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:56.568229 17708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.553198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:46407 (local address 127.0.0.1:57500)
0809 00:44:56.561419 (+ 8221us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:56.561429 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:56.561477 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:56.562875 (+ 1398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:56.562876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:56.562878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:56.562928 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.562929 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.564265 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.564266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.566035 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.566039 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.567947 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.567948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.568018 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.568023 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:44:56.568075 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:44:56.568119 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8204,"mutex_wait_us":13,"thread_start_us":65,"threads_started":1}
I20260809 00:44:56.568295 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.553123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.30.104.194:34379)
0809 00:44:56.559825 (+ 6702us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:56.559830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:56.560130 (+ 300us) server_negotiation.cc:407] Connection header received
0809 00:44:56.560170 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:56.560172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:56.560188 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:56.560289 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:56.560515 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.562321 (+ 1806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.566901 (+ 4580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.567726 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.567811 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.567864 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:56.568169 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 00:44:56.568215 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6661,"thread_start_us":46,"threads_started":1}
I20260809 00:44:56.568719 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.553332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.30.104.193:46407)
0809 00:44:56.553513 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:56.553518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:56.561543 (+ 8025us) server_negotiation.cc:407] Connection header received
0809 00:44:56.561598 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:56.561601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:56.561617 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:56.561685 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:56.563016 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.564157 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.566915 (+ 2758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.567778 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.567855 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.567907 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:56.568596 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 00:44:56.568642 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":56,"threads_started":1}
I20260809 00:44:56.568886 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.553009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:34379 (local address 127.0.0.1:35452)
0809 00:44:56.560001 (+ 6992us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:56.560010 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:56.560047 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:56.560390 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:56.560391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:56.560393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:56.560442 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.560444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.562415 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.562416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.566792 (+ 4376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.566795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.567911 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.567912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.567978 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.567982 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:44:56.568048 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:44:56.568091 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6977,"thread_start_us":81,"threads_started":1}
I20260809 00:44:56.569682 17192 tablet_service.cc:1467] Processing CreateTablet for tablet 28e74925b6484113b9a9e6c42551e3ef (DEFAULT_TABLE table=test-table [id=11e438bbe9a54f269317ba9c6fc1eded]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:44:56.570079 17192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e74925b6484113b9a9e6c42551e3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:56.570192 17384 tablet_service.cc:1467] Processing CreateTablet for tablet 28e74925b6484113b9a9e6c42551e3ef (DEFAULT_TABLE table=test-table [id=11e438bbe9a54f269317ba9c6fc1eded]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:44:56.570555 17384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e74925b6484113b9a9e6c42551e3ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:56.575635 17714 tablet_bootstrap.cc:492] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6: Bootstrap starting.
I20260809 00:44:56.575639 17713 tablet_bootstrap.cc:492] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: Bootstrap starting.
I20260809 00:44:56.576345 17715 tablet_bootstrap.cc:492] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: Bootstrap starting.
I20260809 00:44:56.576459 17713 tablet_bootstrap.cc:654] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:56.576675 17714 tablet_bootstrap.cc:654] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:56.576918 17713 log.cc:824] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:56.577005 17714 log.cc:824] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:56.578006 17715 tablet_bootstrap.cc:654] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:56.578336 17715 log.cc:824] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:56.580185 17715 tablet_bootstrap.cc:492] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: No bootstrap required, opened a new log
I20260809 00:44:56.580307 17715 ts_tablet_manager.cc:1397] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 00:44:56.580636 17713 tablet_bootstrap.cc:492] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: No bootstrap required, opened a new log
I20260809 00:44:56.580706 17713 ts_tablet_manager.cc:1397] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 00:44:56.582998 17715 raft_consensus.cc:348] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.583168 17715 raft_consensus.cc:374] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:56.583210 17715 raft_consensus.cc:729] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4643cadb28c444eba95664893d067613, State: Initialized, Role: FOLLOWER
I20260809 00:44:56.583163 17713 raft_consensus.cc:348] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.583335 17713 raft_consensus.cc:374] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:56.583395 17713 raft_consensus.cc:729] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12da735cd6604daca3c1a8066868ef8c, State: Initialized, Role: FOLLOWER
I20260809 00:44:56.583413 17715 consensus_queue.cc:260] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.583732 17715 ts_tablet_manager.cc:1428] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:44:56.583853 17477 heartbeater.cc:503] Master 127.30.104.254:41677 was elected leader, sending a full tablet report...
I20260809 00:44:56.584704 17713 consensus_queue.cc:260] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.584987 17713 ts_tablet_manager.cc:1428] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 00:44:56.585147 17665 heartbeater.cc:503] Master 127.30.104.254:41677 was elected leader, sending a full tablet report...
I20260809 00:44:56.585646 17714 tablet_bootstrap.cc:492] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6: No bootstrap required, opened a new log
I20260809 00:44:56.585810 17714 ts_tablet_manager.cc:1397] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260809 00:44:56.589007 17714 raft_consensus.cc:348] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.589214 17714 raft_consensus.cc:374] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:56.589250 17714 raft_consensus.cc:729] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365a500341344d59b1bb5eb093e95fd6, State: Initialized, Role: FOLLOWER
I20260809 00:44:56.589397 17714 consensus_queue.cc:260] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.589672 17714 ts_tablet_manager.cc:1428] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 00:44:56.589712 17286 heartbeater.cc:503] Master 127.30.104.254:41677 was elected leader, sending a full tablet report...
W20260809 00:44:56.606464 17288 tablet.cc:2369] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:44:56.690063 17725 raft_consensus.cc:482] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:44:56.690248 17725 raft_consensus.cc:504] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.690665 17725 leader_election.cc:288] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12da735cd6604daca3c1a8066868ef8c (127.30.104.195:33457), 4643cadb28c444eba95664893d067613 (127.30.104.194:34379)
I20260809 00:44:56.694903 17706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.691003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:42863 (local address 127.30.104.195:33457)
0809 00:44:56.691052 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:56.691054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:56.691218 (+ 164us) server_negotiation.cc:407] Connection header received
0809 00:44:56.691423 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:56.691424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:56.691464 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:56.691493 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:56.691741 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.692491 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.694000 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.694518 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.694566 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.694574 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:56.694792 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 00:44:56.694831 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:44:56.694955 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.691303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:58643 (local address 127.30.104.194:34379)
0809 00:44:56.691349 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:56.691352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:56.691426 (+ 74us) server_negotiation.cc:407] Connection header received
0809 00:44:56.691568 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:56.691569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:56.691578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:56.691607 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:44:56.691819 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.692536 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.693981 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.694555 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.694607 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.694615 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:56.694840 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 00:44:56.694883 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:44:56.695209 17729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.691117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:34379 (local address 127.30.104.193:58643)
0809 00:44:56.691366 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:56.691377 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:56.691480 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:56.691676 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:56.691677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:56.691679 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:56.691749 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.691751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.692677 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.692678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.693865 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.693869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.694676 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.694677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.694763 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.694767 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:44:56.695065 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 00:44:56.695112 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":64,"threads_started":1}
I20260809 00:44:56.695634 17726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.690890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:33457 (local address 127.30.104.193:42863)
0809 00:44:56.691138 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:56.691150 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:56.691337 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:56.691577 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:56.691578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:56.691580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:44:56.691664 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.691666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.692601 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.692602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.693854 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:56.693857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.694652 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.694653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:56.694717 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.694721 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:44:56.695392 (+ 671us) client_negotiation.cc:240] Negotiation successful
0809 00:44:56.695552 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":118,"threads_started":1}
I20260809 00:44:56.695698 17416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e74925b6484113b9a9e6c42551e3ef" candidate_uuid: "365a500341344d59b1bb5eb093e95fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4643cadb28c444eba95664893d067613" is_pre_election: true
I20260809 00:44:56.695876 17416 raft_consensus.cc:2454] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365a500341344d59b1bb5eb093e95fd6 in term 0.
I20260809 00:44:56.696133 17588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e74925b6484113b9a9e6c42551e3ef" candidate_uuid: "365a500341344d59b1bb5eb093e95fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12da735cd6604daca3c1a8066868ef8c" is_pre_election: true
I20260809 00:44:56.696167 17124 leader_election.cc:302] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365a500341344d59b1bb5eb093e95fd6, 4643cadb28c444eba95664893d067613; no voters:
I20260809 00:44:56.696269 17588 raft_consensus.cc:2454] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365a500341344d59b1bb5eb093e95fd6 in term 0.
I20260809 00:44:56.696368 17725 raft_consensus.cc:2781] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:44:56.696432 17725 raft_consensus.cc:482] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:44:56.696461 17725 raft_consensus.cc:3037] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:56.697376 17725 raft_consensus.cc:504] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.697532 17725 leader_election.cc:288] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [CANDIDATE]: Term 1 election: Requested vote from peers 12da735cd6604daca3c1a8066868ef8c (127.30.104.195:33457), 4643cadb28c444eba95664893d067613 (127.30.104.194:34379)
I20260809 00:44:56.697806 17588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e74925b6484113b9a9e6c42551e3ef" candidate_uuid: "365a500341344d59b1bb5eb093e95fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12da735cd6604daca3c1a8066868ef8c"
I20260809 00:44:56.697831 17416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e74925b6484113b9a9e6c42551e3ef" candidate_uuid: "365a500341344d59b1bb5eb093e95fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4643cadb28c444eba95664893d067613"
I20260809 00:44:56.697887 17588 raft_consensus.cc:3037] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:56.697953 17416 raft_consensus.cc:3037] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:56.698820 17588 raft_consensus.cc:2454] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365a500341344d59b1bb5eb093e95fd6 in term 1.
I20260809 00:44:56.698908 17416 raft_consensus.cc:2454] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365a500341344d59b1bb5eb093e95fd6 in term 1.
I20260809 00:44:56.699121 17124 leader_election.cc:302] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365a500341344d59b1bb5eb093e95fd6, 4643cadb28c444eba95664893d067613; no voters:
I20260809 00:44:56.699280 17725 raft_consensus.cc:2781] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:44:56.699522 17725 raft_consensus.cc:686] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [term 1 LEADER]: Becoming Leader. State: Replica: 365a500341344d59b1bb5eb093e95fd6, State: Running, Role: LEADER
I20260809 00:44:56.699627 17725 consensus_queue.cc:237] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } }
I20260809 00:44:56.700536 17000 catalog_manager.cc:5495] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 365a500341344d59b1bb5eb093e95fd6 (127.30.104.193). New cstate: current_term: 1 leader_uuid: "365a500341344d59b1bb5eb093e95fd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "365a500341344d59b1bb5eb093e95fd6" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46407 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 } health_report { overall_health: UNKNOWN } } }
I20260809 00:44:56.708463 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.705240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.30.104.193:46407)
0809 00:44:56.705283 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:56.705285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:56.705292 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:44:56.705327 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:56.705328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:56.705347 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:56.705411 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:44:56.705624 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.706314 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.707470 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:56.707776 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:56.707862 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:56.708070 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:44:56.708206 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:44:56.708301 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:56.708355 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 00:44:56.708380 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:44:56.710613 17416 raft_consensus.cc:1261] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 1 FOLLOWER]: Refusing update from remote peer 365a500341344d59b1bb5eb093e95fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:44:56.710767 17588 raft_consensus.cc:1261] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 1 FOLLOWER]: Refusing update from remote peer 365a500341344d59b1bb5eb093e95fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:44:56.711136 17725 consensus_queue.cc:1035] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4643cadb28c444eba95664893d067613" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:44:56.711323 17730 consensus_queue.cc:1035] T 28e74925b6484113b9a9e6c42551e3ef P 365a500341344d59b1bb5eb093e95fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12da735cd6604daca3c1a8066868ef8c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 33457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:44:56.715618 17737 mvcc.cc:205] Tried to move back new op lower bound from 7316423871323893760 to 7316423871282298880. Current Snapshot: MvccSnapshot[applied={T|T < 7316423871323893760}]
I20260809 00:44:56.716356 17736 mvcc.cc:205] Tried to move back new op lower bound from 7316423871323893760 to 7316423871282298880. Current Snapshot: MvccSnapshot[applied={T|T < 7316423871323893760}]
I20260809 00:44:56.717685 16999 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:44:56.717787 16999 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:44:56.718683 16999 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 793b1a3a012741a0accfb0d5f940a245: Sending DeleteTablet for 3 replicas of tablet 28e74925b6484113b9a9e6c42551e3ef
I20260809 00:44:56.719065 17384 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e74925b6484113b9a9e6c42551e3ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:44:56 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:35452
I20260809 00:44:56.719481 17192 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e74925b6484113b9a9e6c42551e3ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:44:56 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:57500
I20260809 00:44:56.719941 17562 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e74925b6484113b9a9e6c42551e3ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:44:56 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:49380
I20260809 00:44:56.720484 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 17056
I20260809 00:44:56.721160 17749 tablet_replica.cc:330] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: stopping tablet replica
I20260809 00:44:56.721261 17749 raft_consensus.cc:2229] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:44:56.721356 17749 raft_consensus.cc:2258] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:44:56.723518 17749 ts_tablet_manager.cc:1905] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:44:56.725184 17752 tablet_replica.cc:330] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: stopping tablet replica
I20260809 00:44:56.725293 17752 raft_consensus.cc:2229] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:44:56.725396 17752 raft_consensus.cc:2258] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:44:56.725852 17752 ts_tablet_manager.cc:1905] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:44:56.727303 17752 ts_tablet_manager.cc:1918] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:44:56.727365 17752 log.cc:1196] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/28e74925b6484113b9a9e6c42551e3ef
I20260809 00:44:56.727656 17752 ts_tablet_manager.cc:1939] T 28e74925b6484113b9a9e6c42551e3ef P 12da735cd6604daca3c1a8066868ef8c: Deleting consensus metadata
I20260809 00:44:56.728053 17749 ts_tablet_manager.cc:1918] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:44:56.728117 17749 log.cc:1196] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/28e74925b6484113b9a9e6c42551e3ef
I20260809 00:44:56.728365 17749 ts_tablet_manager.cc:1939] T 28e74925b6484113b9a9e6c42551e3ef P 4643cadb28c444eba95664893d067613: Deleting consensus metadata
I20260809 00:44:56.729015 16979 catalog_manager.cc:4841] TS 4643cadb28c444eba95664893d067613 (127.30.104.194:34379): tablet 28e74925b6484113b9a9e6c42551e3ef (table test-table [id=11e438bbe9a54f269317ba9c6fc1eded]) successfully deleted
I20260809 00:44:56.731309 16979 catalog_manager.cc:4841] TS 12da735cd6604daca3c1a8066868ef8c (127.30.104.195:33457): tablet 28e74925b6484113b9a9e6c42551e3ef (table test-table [id=11e438bbe9a54f269317ba9c6fc1eded]) successfully deleted
W20260809 00:44:56.737697 16981 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:46407 (error 108)
I20260809 00:44:56.737775 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 17284
I20260809 00:44:56.745924 17705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:56.745742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:46407 (local address 127.0.0.1:57512)
0809 00:44:56.745806 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:56.745873 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:46407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:44:56.746289 16981 catalog_manager.cc:4584] TS 365a500341344d59b1bb5eb093e95fd6 (127.30.104.193:46407): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28e74925b6484113b9a9e6c42551e3ef: Network error: Client connection negotiation failed: client connection to 127.30.104.193:46407: connect: Connection refused (error 111)
I20260809 00:44:56.756250 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 17481
I20260809 00:44:56.794752 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 16951
I20260809 00:44:56.806384 31139 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 00:44:57.153694 31139 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 00:44:57.153837 31139 mini_ranger.cc:66] Stopping Ranger...
I20260809 00:44:57.499271 31139 mini_ranger.cc:68] Stopped Ranger
2026-08-09 00:44:57.499 UTC [15501] LOG: received smart shutdown request
2026-08-09 00:44:57.503 UTC [15501] LOG: background worker "logical replication launcher" (PID 15513) exited with exit code 1
2026-08-09 00:44:57.504 UTC [15506] LOG: shutting down
2026-08-09 00:44:57.689 UTC [15501] LOG: database system is shut down
2026-08-09T00:44:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51669 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:44:57 dist-test-slave-c12x krb5kdc[17770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:44:57 dist-test-slave-c12x krb5kdc[17770](info): set up 2 sockets
Aug 09 00:44:57 dist-test-slave-c12x krb5kdc[17770](info): commencing operation
krb5kdc: starting...
W20260809 00:44:59.054091 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:44:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:44:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:44:59.190550 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:36501
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:40261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:36501,127.30.104.253:39881,127.30.104.252:44605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:44:59.276739 17791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:59.277012 17791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:59.277055 17791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:59.278249 17791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:44:59.278322 17791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:59.278349 17791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:59.278381 17791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:44:59.278415 17791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:44:59.280141 17791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:40261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:36501,127.30.104.253:39881,127.30.104.252:44605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:36501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:59.280450 17791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:59.280696 17791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:59.286041 17799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:59.286058 17797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:59.286331 17791 server_base.cc:1034] running on GCE node
W20260809 00:44:59.286041 17796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:59.286736 17791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:59.287070 17791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:59.288244 17791 hybrid_clock.cc:648] HybridClock initialized: now 1786236299288230 us; error 50 us; skew 500 ppm
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:59.290971 17791 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:44:59.291505 17791 webserver.cc:460] Webserver started at http://127.30.104.254:40937/ using document root <none> and password file <none>
I20260809 00:44:59.291767 17791 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:59.291842 17791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:59.292001 17791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:44:59.293144 17791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "a676f509ad6a437d93280750e1e27bf0"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "33e48fa5d2b39ae92ae7a8bc765c2de5"
server_key_iv: "d5893d27a825910a9da0f05090bd325f"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.293542 17791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "a676f509ad6a437d93280750e1e27bf0"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "33e48fa5d2b39ae92ae7a8bc765c2de5"
server_key_iv: "d5893d27a825910a9da0f05090bd325f"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.295032 17791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:44:59.296041 17809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:59.296363 17791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:44:59.296522 17791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a676f509ad6a437d93280750e1e27bf0"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "33e48fa5d2b39ae92ae7a8bc765c2de5"
server_key_iv: "d5893d27a825910a9da0f05090bd325f"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.296612 17791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:59.318410 17791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:59.321020 17791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:59.321205 17791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:59.325742 17791 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:36501
I20260809 00:44:59.325801 17882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:36501 every 8 connection(s)
I20260809 00:44:59.326206 17791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:44:59.326980 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 17791
I20260809 00:44:59.327069 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:44:59.327291 31139 external_mini_cluster.cc:1428] Setting key 19cea58ff899b0c300cd82965c7607cf
I20260809 00:44:59.327304 17883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:59.329057 17883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.104.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.253@KRBTEST.COM" created.
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:59.344254 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.330020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36501 (local address 127.0.0.1:57182)
0809 00:44:59.330214 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.330262 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.330772 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.331128 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.331131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.331427 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.331758 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.331766 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.333413 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.333414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.334234 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.334237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.335139 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.335140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.335215 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.335775 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.335787 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.340285 (+ 4498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.342465 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.342474 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.342476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.342780 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.343139 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.343140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.343140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.343231 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.343769 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.343775 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.343840 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.343925 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.344070 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":112,"threads_started":1}
I20260809 00:44:59.344348 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.330251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.30.104.254:36501)
0809 00:44:59.330433 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.330438 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.330487 (+ 49us) server_negotiation.cc:407] Connection header received
0809 00:44:59.330937 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.330939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.330963 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.331075 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.331892 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.333293 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.334348 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.334916 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.335034 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.340401 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.340417 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.340417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.340427 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.342358 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.342925 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.342928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.342929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.343061 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.343301 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.343302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.343302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.343655 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.343746 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.343993 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.344164 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260809 00:44:59.345275 17883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:44:59.345738 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.345590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:45950)
0809 00:44:59.345644 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.345707 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 00:44:59.345939 17815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
I20260809 00:44:59.346495 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.346380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:45956)
0809 00:44:59.346436 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.346471 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 00:44:59.346657 17883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:39881: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab.
Entry for principal kudu/127.30.104.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab.
I20260809 00:44:59.358148 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.253:39881
--webserver_interface=127.30.104.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab
--principal=kudu/127.30.104.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:40261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:36501,127.30.104.253:39881,127.30.104.252:44605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:44:59.395929 17883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } attempt: 1
I20260809 00:44:59.396513 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.396354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:45966)
0809 00:44:59.396422 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.396473 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 00:44:59.396912 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.396812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:45982)
0809 00:44:59.396859 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.396890 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:44:59.397017 17883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:39881: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
W20260809 00:44:59.445734 17895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:59.445955 17895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:59.445978 17895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:59.447165 17895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:44:59.447213 17895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:59.447227 17895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:59.447240 17895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:44:59.447253 17895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:44:59.448844 17895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:40261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:36501,127.30.104.253:39881,127.30.104.252:44605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab
--principal=kudu/127.30.104.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.253:39881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.104.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:59.449141 17895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:59.449417 17895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:59.455127 17903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:59.455093 17904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:59.455106 17906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:59.455405 17895 server_base.cc:1034] running on GCE node
I20260809 00:44:59.455713 17895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:59.455994 17895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:59.457134 17895 hybrid_clock.cc:648] HybridClock initialized: now 1786236299457126 us; error 30 us; skew 500 ppm
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:59.459714 17895 init.cc:377] Logged in from keytab as kudu/127.30.104.253@KRBTEST.COM (short username kudu)
I20260809 00:44:59.460234 17895 webserver.cc:460] Webserver started at http://127.30.104.253:38395/ using document root <none> and password file <none>
I20260809 00:44:59.460470 17895 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:59.460518 17895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:59.460623 17895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:44:59.461690 17895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/data/instance:
uuid: "0bea03c732a04a5b9c618d594e49aa17"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "911f97a2c6a8eee58bde43161add6790"
server_key_iv: "a5af65f9dd49e9bd70f0958e1047183f"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.462023 17895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/wal/instance:
uuid: "0bea03c732a04a5b9c618d594e49aa17"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "911f97a2c6a8eee58bde43161add6790"
server_key_iv: "a5af65f9dd49e9bd70f0958e1047183f"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.463415 17895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:44:59.464286 17914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:59.464468 17895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:44:59.464565 17895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/wal
uuid: "0bea03c732a04a5b9c618d594e49aa17"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "911f97a2c6a8eee58bde43161add6790"
server_key_iv: "a5af65f9dd49e9bd70f0958e1047183f"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.464645 17895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:59.465154 17883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } attempt: 2
I20260809 00:44:59.465583 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.465449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:45984)
0809 00:44:59.465512 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.465554 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 00:44:59.465904 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.465805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:45992)
0809 00:44:59.465849 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.465875 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:44:59.465993 17883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:39881: Network error: Client connection negotiation failed: client connection to 127.30.104.253:39881: connect: Connection refused (error 111)
I20260809 00:44:59.488207 17895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:59.490458 17895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:59.490651 17895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:59.495170 17895 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.253:39881
I20260809 00:44:59.495236 17985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.253:39881 every 8 connection(s)
I20260809 00:44:59.495668 17895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
I20260809 00:44:59.496645 17987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:59.498442 17987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:59.505024 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 17895
I20260809 00:44:59.505112 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-1/wal/instance
I20260809 00:44:59.505317 31139 external_mini_cluster.cc:1428] Setting key bb35bd88ec82c4cfa1f4693c30f74dba
I20260809 00:44:59.508069 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.499272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57188 (local address 127.30.104.254:36501)
0809 00:44:59.499335 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.499337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.499691 (+ 354us) server_negotiation.cc:407] Connection header received
0809 00:44:59.500153 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.500154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.500167 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.500212 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.500881 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.501367 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.502358 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.502563 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.502626 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.504890 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.504891 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.504891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.504899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.506420 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.506944 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.506945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.506946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.507023 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.507301 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.507303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.507303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.507521 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.507611 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.507866 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.507966 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:44:59.508138 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.499261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36501 (local address 127.0.0.1:57188)
0809 00:44:59.499551 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.499600 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.500068 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.500297 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.500299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.500604 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.500804 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.500808 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.501476 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.501476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.502277 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.502279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.502651 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.502652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.502764 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.503330 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.503339 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.504773 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.506507 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.506515 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.506517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.506851 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.507110 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.507111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.507112 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.507232 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.507598 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.507604 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.507726 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.507820 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.507948 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":86,"threads_started":1}
I20260809 00:44:59.508702 17987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.104.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.252@KRBTEST.COM" created.
I20260809 00:44:59.517256 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.508999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:46000)
0809 00:44:59.509058 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.509071 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.509128 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.509387 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.509388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.509562 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.509631 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.509633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.510500 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.510502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.511386 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.511389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.511928 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.511930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.512019 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.512407 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.512413 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.513853 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.516015 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.516016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.516017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.516236 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.516495 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.516496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.516497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.516561 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.516952 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.516953 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.516996 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.517074 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.517179 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:44:59.517431 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.509056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46000 (local address 127.30.104.253:39881)
0809 00:44:59.509238 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.509242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.509262 (+ 20us) server_negotiation.cc:407] Connection header received
0809 00:44:59.509291 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.509293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.509303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.509366 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.509713 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.510403 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.511525 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.511844 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.511903 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.513970 (+ 2067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.513980 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.513980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.513990 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.515874 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.516320 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.516325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.516326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.516418 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.516641 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.516642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.516643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.516848 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.516975 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.517128 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.517277 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":153,"thread_start_us":126,"threads_started":1}
I20260809 00:44:59.518136 17987 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:44:59.518512 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.518378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44876)
0809 00:44:59.518434 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.518484 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 00:44:59.518697 17919 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
I20260809 00:44:59.520720 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.520604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44886)
0809 00:44:59.520669 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.520699 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 00:44:59.520860 17987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:44605: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab.
Entry for principal kudu/127.30.104.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab.
I20260809 00:44:59.534252 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.252:44605
--webserver_interface=127.30.104.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab
--principal=kudu/127.30.104.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:40261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:36501,127.30.104.253:39881,127.30.104.252:44605
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:44:59.557219 17883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } attempt: 3
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:44:59.564826 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.557721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46012 (local address 127.30.104.253:39881)
0809 00:44:59.557806 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.557808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.557819 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:44:59.557870 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.557871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.557890 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.557958 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.558476 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.559026 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.559761 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.559987 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.560070 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.562068 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.562070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.562070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.562086 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.563615 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.563994 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.563995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.563996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.564058 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.564271 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.564272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.564272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.564399 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.564511 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.564633 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.564721 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 00:44:59.564826 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.557646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:46012)
0809 00:44:59.557716 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.557730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.557806 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.557976 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.557977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.558290 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.558367 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.558371 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.559120 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.559121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.559669 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.559671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.560071 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.560072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.560166 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.560691 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.560696 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.561957 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.563703 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.563704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.563705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.563895 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.564138 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.564139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.564139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.564204 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.564477 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.564478 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.564531 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.564603 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.564719 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 00:44:59.565438 17883 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:44:59.565850 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.565734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44892)
0809 00:44:59.565792 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.565827 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:44:59.566188 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.566090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44906)
0809 00:44:59.566138 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.566163 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:44:59.566287 17883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:44605: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
I20260809 00:44:59.570128 17987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } attempt: 1
I20260809 00:44:59.570608 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.570457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44908)
0809 00:44:59.570539 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.570579 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 00:44:59.571019 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.570927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44914)
0809 00:44:59.570970 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.570997 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:44:59.571130 17987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:44605: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
I20260809 00:44:59.597465 17883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } attempt: 1
I20260809 00:44:59.598026 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.597870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44926)
0809 00:44:59.597950 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.597995 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 00:44:59.598484 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.598379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44942)
0809 00:44:59.598431 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.598465 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 00:44:59.598613 17883 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:44605: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
W20260809 00:44:59.625008 17996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:44:59.625303 17996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:44:59.625380 17996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:44:59.626704 17996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:44:59.626797 17996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:44:59.626830 17996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:44:59.626865 17996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:44:59.626899 17996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:44:59.628980 17996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:40261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:36501,127.30.104.253:39881,127.30.104.252:44605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab
--principal=kudu/127.30.104.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.252:44605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.104.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:44:59.629325 17996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:44:59.629619 17996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:44:59.635313 18005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:59.635519 17996 server_base.cc:1034] running on GCE node
W20260809 00:44:59.635316 18007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:44:59.635308 18004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:44:59.635888 17996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:44:59.636152 17996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:44:59.637305 17996 hybrid_clock.cc:648] HybridClock initialized: now 1786236299637292 us; error 28 us; skew 500 ppm
I20260809 00:44:59.639304 17987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } attempt: 2
I20260809 00:44:59.639809 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.639660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44958)
0809 00:44:59.639728 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.639771 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:44:59.640213 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.640099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44964)
0809 00:44:59.640158 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.640187 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:44:59.640303 17996 init.cc:377] Logged in from keytab as kudu/127.30.104.252@KRBTEST.COM (short username kudu)
W20260809 00:44:59.640334 17987 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:44605: Network error: Client connection negotiation failed: client connection to 127.30.104.252:44605: connect: Connection refused (error 111)
I20260809 00:44:59.640827 17996 webserver.cc:460] Webserver started at http://127.30.104.252:33435/ using document root <none> and password file <none>
I20260809 00:44:59.641088 17996 fs_manager.cc:359] Metadata directory not provided
I20260809 00:44:59.641157 17996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:44:59.641322 17996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:44:59.642426 17996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/data/instance:
uuid: "25f26489bb704142929eb16d4372325e"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "7256faa63581ef873deb49e6f622b791"
server_key_iv: "7aefcf8ae05d35d8a758b7346341a6a9"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.642956 17996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/wal/instance:
uuid: "25f26489bb704142929eb16d4372325e"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "7256faa63581ef873deb49e6f622b791"
server_key_iv: "7aefcf8ae05d35d8a758b7346341a6a9"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.644471 17996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:44:59.645344 18017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:44:59.645653 17996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:44:59.645777 17996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/wal
uuid: "25f26489bb704142929eb16d4372325e"
format_stamp: "Formatted at 2026-08-09 00:44:59 on dist-test-slave-c12x"
server_key: "7256faa63581ef873deb49e6f622b791"
server_key_iv: "7aefcf8ae05d35d8a758b7346341a6a9"
server_key_version: "encryptionkey@0"
I20260809 00:44:59.645866 17996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:44:59.657389 17996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:44:59.659811 17996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:44:59.660020 17996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:44:59.664312 17996 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.252:44605
I20260809 00:44:59.664346 18097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.252:44605 every 8 connection(s)
I20260809 00:44:59.664716 17996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
I20260809 00:44:59.665686 18098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:44:59.667364 18098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:44:59.670322 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 17996
I20260809 00:44:59.670506 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786236131192310-31139-0/minicluster-data/master-2/wal/instance
I20260809 00:44:59.670732 31139 external_mini_cluster.cc:1428] Setting key 587cd08c1fabc5ad17c163ccdc089dbb
I20260809 00:44:59.673799 17883 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } attempt: 2
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:44:59.678598 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.668208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.30.104.254:36501)
0809 00:44:59.668279 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.668282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.668541 (+ 259us) server_negotiation.cc:407] Connection header received
0809 00:44:59.669032 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.669033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.669044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.669117 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.669769 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.670381 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.672038 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.672354 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.672421 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.675026 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.675027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.675028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.675046 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.676717 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.677250 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.677251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.677252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.677310 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.677654 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.677655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.677656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.677782 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.677866 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.678360 (+ 494us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.678485 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:44:59.679046 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.668195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:36501 (local address 127.0.0.1:57194)
0809 00:44:59.668388 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.668454 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.668933 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.669188 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.669191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.669478 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.669673 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.669677 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.670536 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.670537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.671948 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.671950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.672443 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.672444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.672632 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.673181 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.673190 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.674882 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.676815 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.676822 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.676824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.677141 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.677393 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.677394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.677394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.677562 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.678069 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.678075 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.678196 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.678318 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.678774 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":101,"threads_started":1}
I20260809 00:44:59.680104 18098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:44:59.681164 18103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.671197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44966 (local address 127.30.104.252:44605)
0809 00:44:59.671373 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.671378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.671402 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:44:59.671535 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.671538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.671561 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.671642 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.672305 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.673281 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.674173 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.674654 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.674731 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.677555 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.677563 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.677563 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.677570 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.679594 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.680027 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.680030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.680031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.680091 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.680326 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.680327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.680327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.680511 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.680660 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.680732 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.680852 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
I20260809 00:44:59.681811 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.674065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44968)
0809 00:44:59.674116 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.674127 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.674163 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.674584 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.674585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.674769 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.674811 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.674813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.675745 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.675746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.676342 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.676344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.676744 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.676745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.676808 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.677209 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.677215 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.678772 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.680738 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.680739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.680740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.680957 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.681194 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.681196 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.681196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.681272 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.681561 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.681562 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.681621 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.681670 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.681750 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:44:59.682080 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.674235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44968 (local address 127.30.104.252:44605)
0809 00:44:59.674406 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.674408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.674417 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:44:59.674438 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.674439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.674449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.674555 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.674922 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.675645 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.676454 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.676657 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.676730 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.678883 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.678884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.678884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.678891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.680646 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.681039 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.681040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.681040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.681115 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.681354 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.681355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.681355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.681480 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.681896 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.681928 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.681984 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260809 00:44:59.684089 17883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0: Bootstrap starting.
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:44:59.684878 17883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:59.685243 17883 log.cc:824] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:59.686146 17883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0: No bootstrap required, opened a new log
I20260809 00:44:59.687806 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.680403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:39881 (local address 127.0.0.1:46014)
0809 00:44:59.680445 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.680454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.680487 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.680699 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.680700 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.680893 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.680937 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.680938 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.681828 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.681829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.682628 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.682630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.683193 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.683194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.683256 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.683661 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.683666 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.684999 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.686720 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.686721 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.686721 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.686943 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.687160 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.687160 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.687161 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.687229 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.687486 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.687487 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.687533 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.687612 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.687707 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 00:44:59.687862 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.680490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46014 (local address 127.30.104.253:39881)
0809 00:44:59.680539 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.680541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.680549 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:44:59.680578 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.680578 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.680589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.680647 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.681041 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.681732 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.682731 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.683065 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.683113 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.685095 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.685097 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.685097 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.685106 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.686630 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.687029 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.687029 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.687030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.687092 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.687295 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.687295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.687295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.687393 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.687508 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.687667 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.687747 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:44:59.688315 17883 raft_consensus.cc:348] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
I20260809 00:44:59.688642 17883 raft_consensus.cc:374] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:59.688714 18098 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:44:59.688755 17883 raft_consensus.cc:729] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a676f509ad6a437d93280750e1e27bf0, State: Initialized, Role: FOLLOWER
I20260809 00:44:59.688913 17883 consensus_queue.cc:260] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:44:59.689369 18107 sys_catalog.cc:455] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.689435 17883 sys_catalog.cc:564] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:44:59.689524 18107 sys_catalog.cc:458] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:44:59.692376 18122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:44:59.692512 18122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:44:59.692785 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.682054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57200 (local address 127.30.104.254:36501)
0809 00:44:59.682373 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.682375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.682449 (+ 74us) server_negotiation.cc:407] Connection header received
0809 00:44:59.682529 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.682530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.682538 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.682565 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.683181 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.683884 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.685032 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.685323 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.685398 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.689570 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.689572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.689572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.689580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.691238 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.691675 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.691677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.691677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.691740 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.691947 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.691948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.691949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.692086 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.692223 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.692624 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.692703 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:44:59.697126 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.689008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.30.104.252:44605)
0809 00:44:59.689046 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.689047 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.689056 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:44:59.689124 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.689125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.689136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.689193 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.689490 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.690212 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.691348 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.691669 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.691714 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.693987 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.693988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.693989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.693996 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.695831 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.696237 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.696237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.696238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.696290 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.696506 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.696507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.696507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.696685 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.696735 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.696940 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.697026 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:44:59.694068 17791 master_runner.cc:186] Error getting master registration for 127.30.104.253:39881: OK, instance_id { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" instance_seqno: 1786236299493830 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:44:59.697682 18101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.688966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44976)
0809 00:44:59.689008 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.689017 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.689056 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.689198 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.689199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.689350 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.689385 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.689388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.690309 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.690310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.691251 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.691254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.691790 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.691791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.691839 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.692226 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.692232 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.693895 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.695925 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.695925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.695926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.696157 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.696377 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.696378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.696378 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.696434 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.696781 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.696781 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.696821 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.697475 (+ 654us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.697578 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 00:44:59.697731 17791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:44:59.699466 18098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: Bootstrap starting.
I20260809 00:44:59.700479 18098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:59.700857 18098 log.cc:824] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:59.701843 18098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: No bootstrap required, opened a new log
I20260809 00:44:59.703665 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.693827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46024 (local address 127.30.104.253:39881)
0809 00:44:59.693865 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.693868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.693876 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:44:59.693968 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.693969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.693979 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.694030 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.694681 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.695646 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.696608 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.696893 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.696934 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.699908 (+ 2974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.699910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.699910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.699920 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.701848 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.702315 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.702316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.702317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.702383 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.702598 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.702599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.702599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.702747 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.702940 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.703470 (+ 530us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.703564 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:44:59.704260 18098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
I20260809 00:44:59.704479 18098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:59.704569 18098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f26489bb704142929eb16d4372325e, State: Initialized, Role: FOLLOWER
I20260809 00:44:59.704708 18098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
I20260809 00:44:59.705062 18126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.705121 18098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:44:59.705224 18126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:44:59.705483 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 00:44:59.708192 18145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:44:59.708312 18145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:44:59.709314 17996 master_runner.cc:186] Error getting master registration for 127.30.104.253:39881: OK, instance_id { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" instance_seqno: 1786236299493830 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:44:59.711658 17996 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:44:59.723063 17887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.707235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57210 (local address 127.30.104.254:36501)
0809 00:44:59.707291 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.707294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.711705 (+ 4411us) server_negotiation.cc:407] Connection header received
0809 00:44:59.711857 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.711859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.711874 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.711948 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.712850 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.713617 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.714835 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.715150 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.715229 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.719544 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.719546 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.719546 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.719556 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.721322 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.721832 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.721833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.721833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.721898 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.722124 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.722125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.722125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.722269 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.722365 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.722843 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.722947 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:44:59.724274 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.707963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.30.104.252:44605)
0809 00:44:59.708085 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.708087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.716157 (+ 8070us) server_negotiation.cc:407] Connection header received
0809 00:44:59.716253 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.716254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.716267 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.716297 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.717073 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.717824 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.718851 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.719148 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.719192 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.721235 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.721236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.721236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.721244 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.722997 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.723398 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.723399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.723399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.723473 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.723685 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.723686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.723686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.723812 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.723893 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.724094 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.724178 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 00:44:59.729830 17991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.707594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46028 (local address 127.30.104.253:39881)
0809 00:44:59.707639 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.707641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.716009 (+ 8368us) server_negotiation.cc:407] Connection header received
0809 00:44:59.716067 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.716069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.716087 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.716172 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.718060 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.718877 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.724325 (+ 5448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.724696 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.724744 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.726698 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.726701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.726701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.726710 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.728492 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.728931 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.728932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.728933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.729001 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.729239 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.729240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.729240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.729382 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.729479 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.729641 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.729730 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:44:59.731509 17987 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } attempt: 3
Aug 09 00:44:59 dist-test-slave-c12x krb5kdc[17770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236299, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:44:59.740816 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.731793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:44605 (local address 127.0.0.1:44998)
0809 00:44:59.731848 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:44:59.731860 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:44:59.731899 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:44:59.732184 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:44:59.732185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:44:59.732362 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:44:59.732420 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.732422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.733953 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.733954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.734771 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:44:59.734774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.735311 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.735312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:44:59.735376 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.735907 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:44:59.735912 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:44:59.737575 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:44:59.739701 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.739702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.739702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.739927 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.740144 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:44:59.740144 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:44:59.740145 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:44:59.740202 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:44:59.740489 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:44:59.740490 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:44:59.740538 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 00:44:59.740600 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:44:59.740705 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:44:59.741411 18105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:44:59.731983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.30.104.252:44605)
0809 00:44:59.732048 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:44:59.732050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:44:59.732058 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:44:59.732080 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:44:59.732081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:44:59.732090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:44:59.732152 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:44:59.733020 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.733857 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.734867 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:44:59.735223 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:44:59.735303 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:44:59.737682 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:44:59.737683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:44:59.737684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:44:59.737692 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:44:59.739602 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.740020 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.740021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.740021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.740079 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:44:59.740266 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:44:59.740267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:44:59.740267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:44:59.740404 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:44:59.740463 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:44:59.741234 (+ 771us) server_negotiation.cc:299] Negotiation successful
0809 00:44:59.741319 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:44:59.743535 17987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17: Bootstrap starting.
I20260809 00:44:59.744500 17987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17: Neither blocks nor log segments found. Creating new log.
I20260809 00:44:59.745010 17987 log.cc:824] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17: Log is configured to *not* fsync() on all Append() calls
I20260809 00:44:59.746244 17987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17: No bootstrap required, opened a new log
I20260809 00:44:59.748449 17987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
I20260809 00:44:59.748603 17987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:44:59.748636 17987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bea03c732a04a5b9c618d594e49aa17, State: Initialized, Role: FOLLOWER
I20260809 00:44:59.748847 17987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
I20260809 00:44:59.749305 17987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 00:44:59.751314 18173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:44:59.751500 18173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:44:59.753314 17895 master_runner.cc:263] No leader master found from master 0bea03c732a04a5b9c618d594e49aa17
I20260809 00:44:59.753364 17895 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:44:59.749255 18156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.754262 18156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:44:59.822135 17820 tablet.cc:2369] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:44:59.910758 18024 tablet.cc:2369] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:44:59.963135 18107 raft_consensus.cc:482] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:44:59.963276 18107 raft_consensus.cc:504] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
I20260809 00:44:59.963685 18107 leader_election.cc:288] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bea03c732a04a5b9c618d594e49aa17 (127.30.104.253:39881), 25f26489bb704142929eb16d4372325e (127.30.104.252:44605)
I20260809 00:44:59.964108 18058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a676f509ad6a437d93280750e1e27bf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f26489bb704142929eb16d4372325e" is_pre_election: true
I20260809 00:44:59.964105 17951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a676f509ad6a437d93280750e1e27bf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bea03c732a04a5b9c618d594e49aa17" is_pre_election: true
I20260809 00:44:59.964290 17951 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a676f509ad6a437d93280750e1e27bf0 in term 0.
I20260809 00:44:59.964304 18058 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a676f509ad6a437d93280750e1e27bf0 in term 0.
I20260809 00:44:59.964533 17815 leader_election.cc:302] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bea03c732a04a5b9c618d594e49aa17, a676f509ad6a437d93280750e1e27bf0; no voters:
I20260809 00:44:59.964722 18107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:44:59.964793 18107 raft_consensus.cc:482] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:44:59.964810 18107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:59.965803 18107 raft_consensus.cc:504] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
I20260809 00:44:59.965958 18107 leader_election.cc:288] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [CANDIDATE]: Term 1 election: Requested vote from peers 0bea03c732a04a5b9c618d594e49aa17 (127.30.104.253:39881), 25f26489bb704142929eb16d4372325e (127.30.104.252:44605)
I20260809 00:44:59.966130 17951 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a676f509ad6a437d93280750e1e27bf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bea03c732a04a5b9c618d594e49aa17"
I20260809 00:44:59.966147 18058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a676f509ad6a437d93280750e1e27bf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25f26489bb704142929eb16d4372325e"
I20260809 00:44:59.966212 17951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:59.966233 18058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:44:59.966936 17951 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a676f509ad6a437d93280750e1e27bf0 in term 1.
I20260809 00:44:59.967024 18058 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a676f509ad6a437d93280750e1e27bf0 in term 1.
I20260809 00:44:59.967110 17815 leader_election.cc:302] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bea03c732a04a5b9c618d594e49aa17, a676f509ad6a437d93280750e1e27bf0; no voters:
I20260809 00:44:59.967227 18107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:44:59.967381 18107 raft_consensus.cc:686] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [term 1 LEADER]: Becoming Leader. State: Replica: a676f509ad6a437d93280750e1e27bf0, State: Running, Role: LEADER
I20260809 00:44:59.967516 18107 consensus_queue.cc:237] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } }
I20260809 00:44:59.968084 18176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a676f509ad6a437d93280750e1e27bf0. Latest consensus state: current_term: 1 leader_uuid: "a676f509ad6a437d93280750e1e27bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.968207 18176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:59.968441 18179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:44:59.968658 18179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:44:59.970391 18058 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [term 1 FOLLOWER]: Refusing update from remote peer a676f509ad6a437d93280750e1e27bf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:44:59.970379 17951 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [term 1 FOLLOWER]: Refusing update from remote peer a676f509ad6a437d93280750e1e27bf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:44:59.970736 18107 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:44:59.970857 18176 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:44:59.972127 18126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a676f509ad6a437d93280750e1e27bf0. Latest consensus state: current_term: 1 leader_uuid: "a676f509ad6a437d93280750e1e27bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.972229 18126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:44:59.972611 18156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a676f509ad6a437d93280750e1e27bf0. Latest consensus state: current_term: 1 leader_uuid: "a676f509ad6a437d93280750e1e27bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.972708 18156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:44:59.973184 18176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a676f509ad6a437d93280750e1e27bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.973284 18176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:59.973526 18179 catalog_manager.cc:1347] Generated new cluster ID: 073141dcfb7545f99f2c28e887707477
I20260809 00:44:59.973577 18179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:44:59.973650 18176 sys_catalog.cc:455] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a676f509ad6a437d93280750e1e27bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.973726 18176 sys_catalog.cc:458] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0 [sys.catalog]: This master's current role is: LEADER
I20260809 00:44:59.973876 18156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a676f509ad6a437d93280750e1e27bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.973953 18156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:44:59.974088 18185 mvcc.cc:205] Tried to move back new op lower bound from 7316423884675190784 to 7316423884667691008. Current Snapshot: MvccSnapshot[applied={T|T < 7316423884675190784 or (T in {7316423884675190784})}]
I20260809 00:44:59.974200 18126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a676f509ad6a437d93280750e1e27bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a676f509ad6a437d93280750e1e27bf0" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 36501 } } peers { permanent_uuid: "0bea03c732a04a5b9c618d594e49aa17" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 39881 } } peers { permanent_uuid: "25f26489bb704142929eb16d4372325e" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 44605 } } }
I20260809 00:44:59.974278 18126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:44:59.974330 18182 mvcc.cc:205] Tried to move back new op lower bound from 7316423884675190784 to 7316423884667691008. Current Snapshot: MvccSnapshot[applied={T|T < 7316423884675190784}]
I20260809 00:44:59.991163 18179 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:44:59.991945 18179 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:45:00.003635 18179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a676f509ad6a437d93280750e1e27bf0: Generated new TSK 0
I20260809 00:45:00.003994 18179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:45:00.708890 18145 catalog_manager.cc:1259] Loaded cluster ID: 073141dcfb7545f99f2c28e887707477
I20260809 00:45:00.708982 18145 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: loading cluster ID for follower catalog manager: success
I20260809 00:45:00.709860 18145 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: acquiring CA information for follower catalog manager: success
I20260809 00:45:00.710233 18145 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 25f26489bb704142929eb16d4372325e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:45:00.752063 18173 catalog_manager.cc:1259] Loaded cluster ID: 073141dcfb7545f99f2c28e887707477
I20260809 00:45:00.752151 18173 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17: loading cluster ID for follower catalog manager: success
I20260809 00:45:00.753041 18173 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17: acquiring CA information for follower catalog manager: success
I20260809 00:45:00.753409 18173 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0bea03c732a04a5b9c618d594e49aa17: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:45:01.060154 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 17791
I20260809 00:45:01.071120 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 17895
I20260809 00:45:01.082916 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 17996
2026-08-09T00:45:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3290 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:45:01 dist-test-slave-c12x krb5kdc[18210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:45:01 dist-test-slave-c12x krb5kdc[18210](info): set up 2 sockets
Aug 09 00:45:01 dist-test-slave-c12x krb5kdc[18210](info): commencing operation
krb5kdc: starting...
W20260809 00:45:02.380360 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:45:02 dist-test-slave-c12x krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:45:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:45:02.515228 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44951
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:44951
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:02.601388 18225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:02.601655 18225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:02.601693 18225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:02.602873 18225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:45:02.602954 18225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:02.602980 18225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:02.603010 18225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:45:02.603042 18225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:45:02.604684 18225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:44951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:02.604975 18225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:02.605219 18225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:02.611070 18234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:02.611090 18225 server_base.cc:1034] running on GCE node
W20260809 00:45:02.611054 18236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:02.611047 18239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:02.611680 18225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:02.612061 18225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:02.613270 18225 hybrid_clock.cc:648] HybridClock initialized: now 1786236302613234 us; error 53 us; skew 500 ppm
Aug 09 00:45:02 dist-test-slave-c12x krb5kdc[18210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236302, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:02.616335 18225 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:45:02.616887 18225 webserver.cc:460] Webserver started at http://127.30.104.254:40377/ using document root <none> and password file <none>
I20260809 00:45:02.617166 18225 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:02.617216 18225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:02.617367 18225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:02.618570 18225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "311de5ee1c674e9fab5367dd7c59b500"
format_stamp: "Formatted at 2026-08-09 00:45:02 on dist-test-slave-c12x"
server_key: "90ae5993b682e957753a62ffe7900d75"
server_key_iv: "92503542c8a8e53eff5c0a90d627d60a"
server_key_version: "encryptionkey@0"
I20260809 00:45:02.619020 18225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "311de5ee1c674e9fab5367dd7c59b500"
format_stamp: "Formatted at 2026-08-09 00:45:02 on dist-test-slave-c12x"
server_key: "90ae5993b682e957753a62ffe7900d75"
server_key_iv: "92503542c8a8e53eff5c0a90d627d60a"
server_key_version: "encryptionkey@0"
I20260809 00:45:02.620662 18225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:45:02.621698 18250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:02.621948 18225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:02.622066 18225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "311de5ee1c674e9fab5367dd7c59b500"
format_stamp: "Formatted at 2026-08-09 00:45:02 on dist-test-slave-c12x"
server_key: "90ae5993b682e957753a62ffe7900d75"
server_key_iv: "92503542c8a8e53eff5c0a90d627d60a"
server_key_version: "encryptionkey@0"
I20260809 00:45:02.622145 18225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:02.632205 18225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:02.634547 18225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:02.634912 18225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:02.639245 18225 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:44951
I20260809 00:45:02.639278 18334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:44951 every 8 connection(s)
I20260809 00:45:02.639711 18225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:45:02.640647 18336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:02.641863 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 18225
I20260809 00:45:02.641986 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:45:02.642203 31139 external_mini_cluster.cc:1428] Setting key ba8473b99ca8c37d5f1048d5cdba275f
I20260809 00:45:02.643544 18336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500: Bootstrap starting.
I20260809 00:45:02.644165 18336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:02.644538 18336 log.cc:824] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:02.645349 18336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500: No bootstrap required, opened a new log
I20260809 00:45:02.647708 18336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311de5ee1c674e9fab5367dd7c59b500" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44951 } }
I20260809 00:45:02.647913 18336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:02.647957 18336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 311de5ee1c674e9fab5367dd7c59b500, State: Initialized, Role: FOLLOWER
I20260809 00:45:02.648101 18336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311de5ee1c674e9fab5367dd7c59b500" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44951 } }
I20260809 00:45:02.648183 18336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:45:02.648206 18336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:45:02.648283 18336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:02.648869 18336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311de5ee1c674e9fab5367dd7c59b500" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44951 } }
I20260809 00:45:02.648962 18336 leader_election.cc:302] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 311de5ee1c674e9fab5367dd7c59b500; no voters:
I20260809 00:45:02.649224 18336 leader_election.cc:288] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:45:02.649266 18342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:45:02.649468 18342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [term 1 LEADER]: Becoming Leader. State: Replica: 311de5ee1c674e9fab5367dd7c59b500, State: Running, Role: LEADER
I20260809 00:45:02.649533 18336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:45:02.649605 18342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311de5ee1c674e9fab5367dd7c59b500" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44951 } }
Aug 09 00:45:02 dist-test-slave-c12x krb5kdc[18210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:02.649974 18343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "311de5ee1c674e9fab5367dd7c59b500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311de5ee1c674e9fab5367dd7c59b500" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44951 } } }
I20260809 00:45:02.650108 18343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:02.649964 18342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 311de5ee1c674e9fab5367dd7c59b500. Latest consensus state: current_term: 1 leader_uuid: "311de5ee1c674e9fab5367dd7c59b500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "311de5ee1c674e9fab5367dd7c59b500" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44951 } } }
I20260809 00:45:02.650239 18342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500 [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:02.650529 18350 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:45:02.651089 18350 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:45:02.652899 18350 catalog_manager.cc:1347] Generated new cluster ID: 504ee05454854921a940b5b512ad91c0
I20260809 00:45:02.652956 18350 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:45:02.654256 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:02.642756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42628 (local address 127.30.104.254:44951)
0809 00:45:02.642955 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:02.642972 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:02.642997 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:45:02.643415 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:02.643422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:02.643559 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:02.643714 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:02.644397 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:02.645843 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:02.646632 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:02.647360 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:02.647533 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:02.650413 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:02.650418 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:02.650419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:02.650431 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:02.652548 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:02.652983 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:02.652986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:02.652986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:02.653099 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:02.653321 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:02.653322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:02.653322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:02.653639 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:02.653796 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:02.653911 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:45:02.654055 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260809 00:45:02.655296 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:45:02.670934 18350 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:45:02.671926 18350 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:45:02.683379 18350 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 311de5ee1c674e9fab5367dd7c59b500: Generated new TSK 0
I20260809 00:45:02.683648 18350 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:45:02.709820 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 18225
2026-08-09T00:45:02Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1622 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:45:02 dist-test-slave-c12x krb5kdc[18377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:45:02 dist-test-slave-c12x krb5kdc[18377](info): set up 2 sockets
Aug 09 00:45:02 dist-test-slave-c12x krb5kdc[18377](info): commencing operation
krb5kdc: starting...
W20260809 00:45:03.950093 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:45:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:45:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:45:04.087203 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39737
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:04.174252 18407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:04.174485 18407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:04.174510 18407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:04.175788 18407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:45:04.175837 18407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:04.175853 18407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:04.175865 18407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:45:04.175877 18407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:45:04.177634 18407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:39737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:04.177949 18407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:04.178223 18407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:04.183480 18412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.183678 18407 server_base.cc:1034] running on GCE node
W20260809 00:45:04.183559 18413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:04.183482 18415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.184158 18407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:04.184486 18407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:04.185680 18407 hybrid_clock.cc:648] HybridClock initialized: now 1786236304185657 us; error 41 us; skew 500 ppm
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:04.191704 18407 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:45:04.192183 18407 webserver.cc:460] Webserver started at http://127.30.104.254:43463/ using document root <none> and password file <none>
I20260809 00:45:04.192454 18407 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:04.192528 18407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:04.192674 18407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:04.193814 18407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "b01c68c633bd9b3c3e0b9b91158d40bf"
server_key_iv: "ea562193dd93e4829bff8882f40c2d94"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.194201 18407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "b01c68c633bd9b3c3e0b9b91158d40bf"
server_key_iv: "ea562193dd93e4829bff8882f40c2d94"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.195749 18407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 00:45:04.196626 18426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.196888 18407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:04.196995 18407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "b01c68c633bd9b3c3e0b9b91158d40bf"
server_key_iv: "ea562193dd93e4829bff8882f40c2d94"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.197099 18407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:04.216078 18407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:04.218434 18407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:04.218621 18407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:04.223052 18407 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:39737
I20260809 00:45:04.223133 18500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:39737 every 8 connection(s)
I20260809 00:45:04.223574 18407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:45:04.223791 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 18407
I20260809 00:45:04.223937 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:45:04.224170 31139 external_mini_cluster.cc:1428] Setting key 9a3642ec1997b1161421b1bb3fa76a95
I20260809 00:45:04.224942 18503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:04.228163 18503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Bootstrap starting.
I20260809 00:45:04.228873 18503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:04.229245 18503 log.cc:824] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:04.230136 18503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: No bootstrap required, opened a new log
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:04.232620 18503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:04.232762 18503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:04.232785 18503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f38e31f7fbe4ba7a51a77e295a9e9d7, State: Initialized, Role: FOLLOWER
I20260809 00:45:04.232933 18503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:04.233022 18503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:45:04.233069 18503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:45:04.233119 18503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:04.233662 18503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:04.233791 18503 leader_election.cc:302] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f38e31f7fbe4ba7a51a77e295a9e9d7; no voters:
I20260809 00:45:04.234000 18503 leader_election.cc:288] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:45:04.234033 18511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:45:04.234162 18511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 1 LEADER]: Becoming Leader. State: Replica: 2f38e31f7fbe4ba7a51a77e295a9e9d7, State: Running, Role: LEADER
I20260809 00:45:04.234309 18503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:45:04.234303 18511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:04.234819 18512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } } }
I20260809 00:45:04.234954 18512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:04.235188 18513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f38e31f7fbe4ba7a51a77e295a9e9d7. Latest consensus state: current_term: 1 leader_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } } }
I20260809 00:45:04.235562 18513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:04.235743 18524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:45:04.236239 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.224742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.30.104.254:39737)
0809 00:45:04.224947 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.224963 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.225010 (+ 47us) server_negotiation.cc:407] Connection header received
0809 00:45:04.225489 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.225496 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.225543 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.225657 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:04.226499 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.227525 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.228586 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.229064 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.229161 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.232062 (+ 2901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:04.232066 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:04.232066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:04.232077 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:04.234025 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.234471 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.234474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.234475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.234576 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.234812 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.234813 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.234813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.235182 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:04.235382 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.235896 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.236043 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":108,"threads_started":1}
I20260809 00:45:04.236272 18524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:45:04.238042 18524 catalog_manager.cc:1347] Generated new cluster ID: 645ee976489d46d290837a69059f191a
I20260809 00:45:04.238097 18524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:45:04.249286 18524 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:45:04.250007 18524 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:45:04.252382 18524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Generated new TSK 0
I20260809 00:45:04.252562 18524 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:45:04.284715 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39737
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:04.371066 18545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:04.371320 18545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:04.371361 18545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:04.372563 18545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:04.372640 18545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:04.372687 18545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:45:04.374213 18545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:04.374506 18545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:04.374758 18545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:04.380038 18550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:04.380044 18552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.380077 18545 server_base.cc:1034] running on GCE node
W20260809 00:45:04.380048 18556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.380470 18545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:04.380755 18545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:04.381908 18545 hybrid_clock.cc:648] HybridClock initialized: now 1786236304381894 us; error 33 us; skew 500 ppm
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:04.384632 18545 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:45:04.385064 18545 webserver.cc:460] Webserver started at http://127.30.104.193:39947/ using document root <none> and password file <none>
I20260809 00:45:04.385268 18545 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:04.385316 18545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:04.385439 18545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:04.386451 18545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "49c61f1504ec4974b5af69e56954499d"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "236dd8043456ab8fc0b71ad942d50848"
server_key_iv: "b0d797443be52ca322a9c594632b19e3"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.386773 18545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "49c61f1504ec4974b5af69e56954499d"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "236dd8043456ab8fc0b71ad942d50848"
server_key_iv: "b0d797443be52ca322a9c594632b19e3"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.388226 18545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:45:04.389142 18566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.389382 18545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:45:04.389604 18545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "49c61f1504ec4974b5af69e56954499d"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "236dd8043456ab8fc0b71ad942d50848"
server_key_iv: "b0d797443be52ca322a9c594632b19e3"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.389712 18545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:04.401499 18545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:04.403827 18545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:04.404023 18545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:04.404289 18545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:04.404639 18545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:04.404699 18545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.404749 18545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:04.404778 18545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.411360 18545 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:40215
I20260809 00:45:04.411418 18735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:40215 every 8 connection(s)
I20260809 00:45:04.411818 18545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:04.421792 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 18545
I20260809 00:45:04.421923 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:45:04.422135 31139 external_mini_cluster.cc:1428] Setting key 0947f22e1e7c81a5ea9d30f368ff2262
I20260809 00:45:04.422458 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.412615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:46091 (local address 127.30.104.254:39737)
0809 00:45:04.412680 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.412682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.412896 (+ 214us) server_negotiation.cc:407] Connection header received
0809 00:45:04.413385 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.413386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.413395 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.413464 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:04.414474 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.415410 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.416662 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.416876 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.416940 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.419285 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:04.419287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:04.419287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:04.419294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:04.420754 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.421169 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.421170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.421170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.421223 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.421501 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.421503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.421503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.421630 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:04.421694 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.422196 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.422300 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 00:45:04.422567 18740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.412568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:46091)
0809 00:45:04.412752 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.412795 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:04.413300 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:04.413518 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:04.413521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:04.414129 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:04.414338 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.414343 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.415576 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.415578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.416562 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.416566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.416970 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.416971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.417090 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.417649 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:04.417658 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:04.419182 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:04.420847 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:04.420855 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:04.420856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:04.421089 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:04.421281 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:04.421281 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:04.421282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:04.421384 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:04.421809 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:04.421817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:04.422013 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 00:45:04.422140 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 00:45:04.422325 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260809 00:45:04.423164 18737 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39737
I20260809 00:45:04.423283 18737 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:04.423537 18737 heartbeater.cc:511] Master 127.30.104.254:39737 requested a full tablet report, sending...
I20260809 00:45:04.424139 18448 ts_manager.cc:194] Registered new tserver with Master: 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215)
I20260809 00:45:04.424882 18448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:46091
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:45:04.450284 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39737
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:04.533799 18759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:04.534039 18759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:04.534065 18759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:04.535420 18759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:04.535530 18759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:04.535573 18759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:45:04.537529 18759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:04.537810 18759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:04.538081 18759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:04.543334 18768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:04.543387 18766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.543758 18759 server_base.cc:1034] running on GCE node
W20260809 00:45:04.543334 18765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.544064 18759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:04.544286 18759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:04.545485 18759 hybrid_clock.cc:648] HybridClock initialized: now 1786236304545483 us; error 45 us; skew 500 ppm
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:04.548444 18759 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:45:04.549016 18759 webserver.cc:460] Webserver started at http://127.30.104.194:42865/ using document root <none> and password file <none>
I20260809 00:45:04.549311 18759 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:04.549400 18759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:04.549566 18759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:04.550778 18759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "3bd155548e1749ac9b690c8b292f9043"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "4771af61b5f69ce119ce4b5435e12594"
server_key_iv: "a0f68754c4259288c4f535066b203959"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.551194 18759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "3bd155548e1749ac9b690c8b292f9043"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "4771af61b5f69ce119ce4b5435e12594"
server_key_iv: "a0f68754c4259288c4f535066b203959"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.552862 18759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:45:04.553753 18778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.553972 18759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:04.554088 18759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "3bd155548e1749ac9b690c8b292f9043"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "4771af61b5f69ce119ce4b5435e12594"
server_key_iv: "a0f68754c4259288c4f535066b203959"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.554171 18759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:04.573347 18759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:04.575704 18759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:04.575891 18759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:04.576192 18759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:04.576653 18759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:04.576690 18759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.576714 18759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:04.576727 18759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.582952 18759 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:41313
I20260809 00:45:04.582998 18951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:41313 every 8 connection(s)
I20260809 00:45:04.583349 18759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:45:04.586575 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 18759
I20260809 00:45:04.586711 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:45:04.586908 31139 external_mini_cluster.cc:1428] Setting key 6d5b854b9fdcb6cb33e4617e1fcb0fbe
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:04.594396 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.584263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:36163 (local address 127.30.104.254:39737)
0809 00:45:04.584339 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.584342 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.584590 (+ 248us) server_negotiation.cc:407] Connection header received
0809 00:45:04.585100 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.585101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.585113 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.585170 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:04.586175 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.586676 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.587903 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.588209 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.588288 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.590853 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:04.590854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:04.590855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:04.590863 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:04.592730 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.593182 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.593183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.593184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.593251 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.593500 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.593501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.593502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.593667 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:04.593750 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.594154 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.594252 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:45:04.594439 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.584241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:36163)
0809 00:45:04.584457 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.584498 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:04.585003 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:04.585255 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:04.585259 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:04.585874 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:04.586084 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.586088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.586821 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.586822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.587814 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.587817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.588324 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.588325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.588445 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.589056 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:04.589065 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:04.590741 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:04.592821 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:04.592828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:04.592830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:04.593094 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:04.593323 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:04.593324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:04.593324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:04.593430 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:04.593774 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:04.593780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:04.594001 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 00:45:04.594103 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 00:45:04.594241 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":112,"threads_started":1}
I20260809 00:45:04.594983 18953 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39737
I20260809 00:45:04.595100 18953 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:04.595319 18953 heartbeater.cc:511] Master 127.30.104.254:39737 requested a full tablet report, sending...
I20260809 00:45:04.595814 18448 ts_manager.cc:194] Registered new tserver with Master: 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313)
I20260809 00:45:04.596210 18448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:36163
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:45:04.618773 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39737
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:04.703810 18962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:04.704066 18962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:04.704114 18962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:04.705271 18962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:04.705356 18962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:04.705407 18962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:45:04.706931 18962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:04.707221 18962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:04.707558 18962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:04.713033 18969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.713137 18962 server_base.cc:1034] running on GCE node
W20260809 00:45:04.712978 18970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:04.713244 18972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.713485 18962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:04.713802 18962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:04.715000 18962 hybrid_clock.cc:648] HybridClock initialized: now 1786236304714984 us; error 41 us; skew 500 ppm
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:04.718106 18962 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:45:04.718607 18962 webserver.cc:460] Webserver started at http://127.30.104.195:38595/ using document root <none> and password file <none>
I20260809 00:45:04.718844 18962 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:04.718899 18962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:04.718997 18962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:04.720237 18962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "13c15921d4fd4edf8a07163bbf7cfb5b"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "022ca68bde02b97f286d1374d004a37e"
server_key_iv: "e992a961427f948fcb7275bd0a7c1318"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.720640 18962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "13c15921d4fd4edf8a07163bbf7cfb5b"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "022ca68bde02b97f286d1374d004a37e"
server_key_iv: "e992a961427f948fcb7275bd0a7c1318"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.722043 18962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:45:04.722790 18982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.722991 18962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:04.723083 18962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "13c15921d4fd4edf8a07163bbf7cfb5b"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "022ca68bde02b97f286d1374d004a37e"
server_key_iv: "e992a961427f948fcb7275bd0a7c1318"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.723140 18962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:04.733321 18962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:04.735965 18962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:04.736135 18962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:04.736370 18962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:04.736694 18962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:04.736728 18962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.736750 18962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:04.736793 18962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.742518 18962 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:37267
I20260809 00:45:04.742581 19150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:37267 every 8 connection(s)
I20260809 00:45:04.742920 18962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:45:04.744993 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 18962
I20260809 00:45:04.745114 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:45:04.745294 31139 external_mini_cluster.cc:1428] Setting key 28068ca1f42893550247395efa2e8954
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:04.752681 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.743803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:42425 (local address 127.30.104.254:39737)
0809 00:45:04.743867 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.743869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.744104 (+ 235us) server_negotiation.cc:407] Connection header received
0809 00:45:04.744618 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.744620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.744632 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.744698 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:04.745763 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.746277 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.747325 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.747617 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.747704 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.749837 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:04.749839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:04.749839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:04.749847 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:04.751103 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.751542 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.751544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.751544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.751611 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.751865 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.751866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.751866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.751999 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:04.752083 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.752490 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.752598 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:45:04.752813 19154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.743777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:42425)
0809 00:45:04.743974 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.744009 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:04.744517 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:04.744754 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:04.744757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:04.745448 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:04.745658 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.745664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.746380 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.746381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.747227 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.747230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.747717 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.747718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.747851 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.748301 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:04.748309 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:04.749734 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:04.751193 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:04.751200 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:04.751202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:04.751424 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:04.751687 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:04.751688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:04.751688 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:04.751796 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:04.752072 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:04.752079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:04.752247 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 00:45:04.752385 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 00:45:04.752611 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":116,"threads_started":1}
I20260809 00:45:04.753278 19152 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39737
I20260809 00:45:04.753394 19152 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:04.753592 19152 heartbeater.cc:511] Master 127.30.104.254:39737 requested a full tablet report, sending...
I20260809 00:45:04.754047 18448 ts_manager.cc:194] Registered new tserver with Master: 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267)
I20260809 00:45:04.754439 18448 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:42425
I20260809 00:45:04.757076 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:45:04.764842 18507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.758490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38172 (local address 127.30.104.254:39737)
0809 00:45:04.758548 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.758550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.758687 (+ 137us) server_negotiation.cc:407] Connection header received
0809 00:45:04.758751 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.758751 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.758759 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.758810 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:04.759499 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.759997 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.760775 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.760998 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.761075 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.762300 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:04.762301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:04.762302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:04.762308 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:04.763738 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.764092 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.764093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.764093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.764146 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:04.764322 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:04.764322 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:04.764323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:04.764462 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:04.764541 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.764711 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.764773 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:45:04.766976 18448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38172:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:45:04.768703 18448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:45:04.776304 19172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.771895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.30.104.194:41313)
0809 00:45:04.772107 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.772113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.772126 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:45:04.772167 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.772169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.772185 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.772301 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:04.772609 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.773831 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.775101 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.775952 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.776041 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.776097 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.776199 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.776235 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":92,"threads_started":1}
I20260809 00:45:04.776289 19167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.771328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:40215 (local address 127.0.0.1:48584)
0809 00:45:04.771565 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.771609 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:04.771683 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:04.772651 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:04.772651 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:04.772653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:04.772713 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.772715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.773634 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.773635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.775055 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.775058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.775903 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.775904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.776014 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.776033 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 00:45:04.776112 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:45:04.776180 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":77,"thread_start_us":63,"threads_started":1}
I20260809 00:45:04.776289 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.771635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:41313 (local address 127.0.0.1:46660)
0809 00:45:04.771777 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.771790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:04.771870 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:04.772492 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:04.772493 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:04.772495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:04.772527 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.772528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.773976 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.773977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.775002 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.775004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.776037 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.776038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.776104 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.776108 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:45:04.776177 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 00:45:04.776210 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":92,"threads_started":1}
I20260809 00:45:04.776577 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.771332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:37267 (local address 127.0.0.1:44024)
0809 00:45:04.771618 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.771629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:04.771749 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:04.772305 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:04.772309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:04.772311 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:04.772390 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.772392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.773955 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.773956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.775107 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.775109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.776324 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.776325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.776382 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.776386 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:45:04.776448 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:45:04.776494 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":62,"thread_start_us":183,"threads_started":1}
I20260809 00:45:04.776832 19170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.771531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48584 (local address 127.30.104.193:40215)
0809 00:45:04.772463 (+ 932us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.772468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.772479 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:45:04.772521 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.772523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.772540 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.772651 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:04.772796 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.773519 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.775177 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.775784 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.775903 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.775968 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.776700 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.776744 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":890,"thread_start_us":99,"threads_started":1}
I20260809 00:45:04.776831 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.771415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44024 (local address 127.30.104.195:37267)
0809 00:45:04.772109 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.772112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.772147 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:45:04.772191 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.772193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.772210 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.772295 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:04.772507 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.773831 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.775181 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.775967 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.776041 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.776096 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.776694 (+ 598us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.776744 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":124,"threads_started":1}
I20260809 00:45:04.777642 18634 tablet_service.cc:1467] Processing CreateTablet for tablet 6082296001e74ca886f51b9761ade684 (DEFAULT_TABLE table=test-table [id=ee4f84cdba5d43d080800328893761ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:04.777804 19054 tablet_service.cc:1467] Processing CreateTablet for tablet 6082296001e74ca886f51b9761ade684 (DEFAULT_TABLE table=test-table [id=ee4f84cdba5d43d080800328893761ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:04.778084 18634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6082296001e74ca886f51b9761ade684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:04.778220 19054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6082296001e74ca886f51b9761ade684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:04.779925 18853 tablet_service.cc:1467] Processing CreateTablet for tablet 6082296001e74ca886f51b9761ade684 (DEFAULT_TABLE table=test-table [id=ee4f84cdba5d43d080800328893761ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:04.780274 18853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6082296001e74ca886f51b9761ade684. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:04.781157 19173 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: Bootstrap starting.
I20260809 00:45:04.781392 19174 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Bootstrap starting.
I20260809 00:45:04.781929 19173 tablet_bootstrap.cc:654] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:04.782037 19174 tablet_bootstrap.cc:654] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:04.782636 19173 log.cc:824] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:04.782636 19174 log.cc:824] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:04.783350 19183 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Bootstrap starting.
I20260809 00:45:04.783708 19173 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: No bootstrap required, opened a new log
I20260809 00:45:04.783794 19173 ts_tablet_manager.cc:1397] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:45:04.784103 19183 tablet_bootstrap.cc:654] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:04.784319 19174 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: No bootstrap required, opened a new log
I20260809 00:45:04.784407 19174 ts_tablet_manager.cc:1397] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:45:04.784665 19183 log.cc:824] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:04.785514 19183 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: No bootstrap required, opened a new log
I20260809 00:45:04.785583 19183 ts_tablet_manager.cc:1397] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:45:04.786078 19173 raft_consensus.cc:348] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.786263 19173 raft_consensus.cc:374] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:04.786302 19173 raft_consensus.cc:729] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c15921d4fd4edf8a07163bbf7cfb5b, State: Initialized, Role: FOLLOWER
I20260809 00:45:04.786227 19174 raft_consensus.cc:348] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.786427 19174 raft_consensus.cc:374] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:04.786477 19174 raft_consensus.cc:729] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c61f1504ec4974b5af69e56954499d, State: Initialized, Role: FOLLOWER
I20260809 00:45:04.786469 19173 consensus_queue.cc:260] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.786634 19174 consensus_queue.cc:260] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.786801 19173 ts_tablet_manager.cc:1428] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:45:04.786970 19174 ts_tablet_manager.cc:1428] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:45:04.786981 19152 heartbeater.cc:503] Master 127.30.104.254:39737 was elected leader, sending a full tablet report...
I20260809 00:45:04.787151 18737 heartbeater.cc:503] Master 127.30.104.254:39737 was elected leader, sending a full tablet report...
I20260809 00:45:04.787781 19183 raft_consensus.cc:348] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.787979 19183 raft_consensus.cc:374] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:04.788033 19183 raft_consensus.cc:729] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd155548e1749ac9b690c8b292f9043, State: Initialized, Role: FOLLOWER
I20260809 00:45:04.788167 19183 consensus_queue.cc:260] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.788417 19183 ts_tablet_manager.cc:1428] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 00:45:04.788487 18953 heartbeater.cc:503] Master 127.30.104.254:39737 was elected leader, sending a full tablet report...
I20260809 00:45:04.819576 19189 raft_consensus.cc:482] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:04.819705 19189 raft_consensus.cc:504] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.820182 19189 leader_election.cc:288] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267), 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215)
I20260809 00:45:04.822974 18958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.820363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:37267 (local address 127.30.104.194:54683)
0809 00:45:04.820410 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.820424 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:04.820468 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:04.820654 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:04.820656 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:04.820658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:04.820734 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.820737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.821404 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.821405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.822127 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.822129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.822704 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.822705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.822789 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.822792 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:45:04.822854 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:45:04.822900 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 00:45:04.822981 19169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.820437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:54683 (local address 127.30.104.195:37267)
0809 00:45:04.820513 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.820516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.820532 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:45:04.820556 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.820556 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.820564 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.820630 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:04.820810 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.821301 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.822224 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.822610 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.822695 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.822704 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.822877 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.822923 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:45:04.823316 19170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.820464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:35069 (local address 127.30.104.193:40215)
0809 00:45:04.820511 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.820512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.820813 (+ 301us) server_negotiation.cc:407] Connection header received
0809 00:45:04.820888 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.820889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.820906 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.820966 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:04.821121 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.821599 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.822528 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.822933 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.823021 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.823029 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.823205 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.823249 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:45:04.823362 19084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6082296001e74ca886f51b9761ade684" candidate_uuid: "3bd155548e1749ac9b690c8b292f9043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" is_pre_election: true
I20260809 00:45:04.823550 19084 raft_consensus.cc:2454] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd155548e1749ac9b690c8b292f9043 in term 0.
I20260809 00:45:04.823571 19190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.820367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:40215 (local address 127.30.104.194:35069)
0809 00:45:04.820723 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.820735 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:04.820817 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:04.820969 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:04.820970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:04.820971 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:04.821047 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.821048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.821687 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.821689 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.822431 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:04.822433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.823021 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.823022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:04.823104 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.823109 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:45:04.823427 (+ 318us) client_negotiation.cc:240] Negotiation successful
0809 00:45:04.823500 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":154,"thread_start_us":136,"threads_started":1}
I20260809 00:45:04.823805 18782 leader_election.cc:302] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043; no voters:
I20260809 00:45:04.823935 18660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6082296001e74ca886f51b9761ade684" candidate_uuid: "3bd155548e1749ac9b690c8b292f9043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c61f1504ec4974b5af69e56954499d" is_pre_election: true
I20260809 00:45:04.823980 19189 raft_consensus.cc:2781] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:45:04.824065 19189 raft_consensus.cc:482] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:45:04.824105 19189 raft_consensus.cc:3037] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:04.824112 18660 raft_consensus.cc:2454] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd155548e1749ac9b690c8b292f9043 in term 0.
I20260809 00:45:04.824934 19189 raft_consensus.cc:504] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.825084 19189 leader_election.cc:288] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [CANDIDATE]: Term 1 election: Requested vote from peers 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267), 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215)
I20260809 00:45:04.825266 19084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6082296001e74ca886f51b9761ade684" candidate_uuid: "3bd155548e1749ac9b690c8b292f9043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b"
I20260809 00:45:04.825310 18660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6082296001e74ca886f51b9761ade684" candidate_uuid: "3bd155548e1749ac9b690c8b292f9043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c61f1504ec4974b5af69e56954499d"
I20260809 00:45:04.825368 19084 raft_consensus.cc:3037] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:04.825407 18660 raft_consensus.cc:3037] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:04.826116 19084 raft_consensus.cc:2454] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd155548e1749ac9b690c8b292f9043 in term 1.
I20260809 00:45:04.826215 18660 raft_consensus.cc:2454] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd155548e1749ac9b690c8b292f9043 in term 1.
I20260809 00:45:04.826282 18782 leader_election.cc:302] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043; no voters:
I20260809 00:45:04.826375 19189 raft_consensus.cc:2781] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:45:04.826534 19189 raft_consensus.cc:686] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd155548e1749ac9b690c8b292f9043, State: Running, Role: LEADER
I20260809 00:45:04.826689 19189 consensus_queue.cc:237] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:04.827593 18447 catalog_manager.cc:5495] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194). New cstate: current_term: 1 leader_uuid: "3bd155548e1749ac9b690c8b292f9043" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } health_report { overall_health: UNKNOWN } } }
W20260809 00:45:04.833947 18955 tablet.cc:2369] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:45:04.835173 19172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.832619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.30.104.194:41313)
0809 00:45:04.832665 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:04.832667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:04.832677 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:45:04.832734 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:04.832735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:04.832753 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:04.832845 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:45:04.832952 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.833418 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.834373 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:04.834586 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:04.834662 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:04.834827 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:45:04.834930 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:45:04.835018 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:04.835060 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 00:45:04.835080 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:45:04.837072 18660 raft_consensus.cc:1261] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Refusing update from remote peer 3bd155548e1749ac9b690c8b292f9043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:45:04.837157 19084 raft_consensus.cc:1261] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 1 FOLLOWER]: Refusing update from remote peer 3bd155548e1749ac9b690c8b292f9043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:45:04.837440 19191 consensus_queue.cc:1035] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:45:04.837598 19189 consensus_queue.cc:1035] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:45:04.839943 19197 mvcc.cc:205] Tried to move back new op lower bound from 7316423904609689600 to 7316423904570851328. Current Snapshot: MvccSnapshot[applied={T|T < 7316423904609689600}]
I20260809 00:45:04.840148 19198 mvcc.cc:205] Tried to move back new op lower bound from 7316423904609689600 to 7316423904570851328. Current Snapshot: MvccSnapshot[applied={T|T < 7316423904609689600}]
I20260809 00:45:04.840725 19199 mvcc.cc:205] Tried to move back new op lower bound from 7316423904609689600 to 7316423904570851328. Current Snapshot: MvccSnapshot[applied={T|T < 7316423904609689600}]
I20260809 00:45:04.842338 18447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:45:04.842543 18447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38172:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:45:04.843346 18447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Sending DeleteTablet for 3 replicas of tablet 6082296001e74ca886f51b9761ade684
I20260809 00:45:04.843746 19054 tablet_service.cc:1514] Processing DeleteTablet for tablet 6082296001e74ca886f51b9761ade684 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:04 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:44024
I20260809 00:45:04.843775 18634 tablet_service.cc:1514] Processing DeleteTablet for tablet 6082296001e74ca886f51b9761ade684 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:04 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:48584
I20260809 00:45:04.843782 18853 tablet_service.cc:1514] Processing DeleteTablet for tablet 6082296001e74ca886f51b9761ade684 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:04 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:46660
I20260809 00:45:04.843855 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 18545
I20260809 00:45:04.850373 19215 tablet_replica.cc:330] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: stopping tablet replica
I20260809 00:45:04.850544 19215 raft_consensus.cc:2229] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:45:04.850704 19215 raft_consensus.cc:2258] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:04.851156 19215 ts_tablet_manager.cc:1905] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:04.853416 19215 ts_tablet_manager.cc:1918] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:04.853504 19215 log.cc:1196] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/6082296001e74ca886f51b9761ade684
I20260809 00:45:04.853833 19215 ts_tablet_manager.cc:1939] T 6082296001e74ca886f51b9761ade684 P 3bd155548e1749ac9b690c8b292f9043: Deleting consensus metadata
I20260809 00:45:04.854386 18431 catalog_manager.cc:4841] TS 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313): tablet 6082296001e74ca886f51b9761ade684 (table test-table [id=ee4f84cdba5d43d080800328893761ca]) successfully deleted
W20260809 00:45:04.859932 18430 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:40215 (error 108)
I20260809 00:45:04.860160 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 18759
I20260809 00:45:04.860618 19165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:04.860450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:40215 (local address 127.0.0.1:48590)
0809 00:45:04.860511 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:04.860547 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 00:45:04.862212 18430 catalog_manager.cc:4584] TS 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6082296001e74ca886f51b9761ade684: Network error: Client connection negotiation failed: client connection to 127.30.104.193:40215: connect: Connection refused (error 111)
I20260809 00:45:04.862249 19213 tablet_replica.cc:330] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: stopping tablet replica
I20260809 00:45:04.862380 19213 raft_consensus.cc:2229] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:45:04.862509 19213 raft_consensus.cc:2258] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:04.864109 19213 ts_tablet_manager.cc:1905] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:04.866487 19213 ts_tablet_manager.cc:1918] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:04.866575 19213 log.cc:1196] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/6082296001e74ca886f51b9761ade684
I20260809 00:45:04.866886 19213 ts_tablet_manager.cc:1939] T 6082296001e74ca886f51b9761ade684 P 13c15921d4fd4edf8a07163bbf7cfb5b: Deleting consensus metadata
I20260809 00:45:04.867517 18431 catalog_manager.cc:4841] TS 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267): tablet 6082296001e74ca886f51b9761ade684 (table test-table [id=ee4f84cdba5d43d080800328893761ca]) successfully deleted
I20260809 00:45:04.871191 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 18962
I20260809 00:45:04.880092 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 18407
I20260809 00:45:04.891564 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39737
--webserver_interface=127.30.104.254
--webserver_port=43463
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:04.975571 19218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:04.975795 19218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:04.975818 19218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:04.976930 19218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:45:04.976969 19218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:04.976982 19218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:04.976995 19218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:45:04.977007 19218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:45:04.978459 19218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:39737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:04.978680 19218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:04.978869 19218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:04.988649 19230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:04.988651 19227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:04.988653 19228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:04.989017 19218 server_base.cc:1034] running on GCE node
I20260809 00:45:04.989360 19218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:04.989730 19218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:04.990888 19218 hybrid_clock.cc:648] HybridClock initialized: now 1786236304990875 us; error 30 us; skew 500 ppm
Aug 09 00:45:04 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:04.993628 19218 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:45:04.994110 19218 webserver.cc:460] Webserver started at http://127.30.104.254:43463/ using document root <none> and password file <none>
I20260809 00:45:04.994369 19218 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:04.994460 19218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:04.996158 19218 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:45:04.997077 19242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:04.997300 19218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:04.997426 19218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "b01c68c633bd9b3c3e0b9b91158d40bf"
server_key_iv: "ea562193dd93e4829bff8882f40c2d94"
server_key_version: "encryptionkey@0"
I20260809 00:45:04.997910 19218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:05.012068 19218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:05.014616 19218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:05.014823 19218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:05.019199 19218 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:39737
I20260809 00:45:05.019235 19318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:39737 every 8 connection(s)
I20260809 00:45:05.019697 19218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:45:05.021395 19319 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:45:05.022117 19319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Bootstrap starting.
I20260809 00:45:05.027598 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 19218
I20260809 00:45:05.028023 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:40215
--local_ip_for_outbound_sockets=127.30.104.193
--tserver_master_addrs=127.30.104.254:39737
--webserver_port=39947
--webserver_interface=127.30.104.193
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:45:05.031076 19319 log.cc:824] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:05.033210 19319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:45:05.033561 19319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Bootstrap complete.
I20260809 00:45:05.036063 19319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:05.036299 19319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f38e31f7fbe4ba7a51a77e295a9e9d7, State: Initialized, Role: FOLLOWER
I20260809 00:45:05.036473 19319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:05.036545 19319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:45:05.036614 19319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:45:05.036661 19319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:45:05.037734 19319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:05.037827 19319 leader_election.cc:302] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f38e31f7fbe4ba7a51a77e295a9e9d7; no voters:
I20260809 00:45:05.038038 19319 leader_election.cc:288] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:45:05.038084 19326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:45:05.038195 19326 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 2 LEADER]: Becoming Leader. State: Replica: 2f38e31f7fbe4ba7a51a77e295a9e9d7, State: Running, Role: LEADER
I20260809 00:45:05.038295 19326 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:05.038334 19319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:45:05.038589 19328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } } }
I20260809 00:45:05.038894 19328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:05.039196 19332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f38e31f7fbe4ba7a51a77e295a9e9d7. Latest consensus state: current_term: 2 leader_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } } }
I20260809 00:45:05.039243 19347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:45:05.039376 19332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:05.040251 19347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:45:05.040361 19347 catalog_manager.cc:1259] Loaded cluster ID: 645ee976489d46d290837a69059f191a
I20260809 00:45:05.040413 19347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:45:05.040657 19347 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 00:45:05.045610 19347 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 00:45:05.048362 19347 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:45:05.048617 19347 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Loaded TSK: 0
I20260809 00:45:05.048857 19347 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:45:05.120779 19324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:05.121052 19324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:05.121099 19324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:05.122339 19324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:05.122421 19324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:05.122473 19324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:45:05.124264 19324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:40215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=39947
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:05.124579 19324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:05.124850 19324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:05.131868 19374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.131871 19372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.131868 19378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:05.132131 19324 server_base.cc:1034] running on GCE node
I20260809 00:45:05.132340 19324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:05.132643 19324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:05.133831 19324 hybrid_clock.cc:648] HybridClock initialized: now 1786236305133810 us; error 39 us; skew 500 ppm
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:05.136936 19324 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:45:05.137413 19324 webserver.cc:460] Webserver started at http://127.30.104.193:39947/ using document root <none> and password file <none>
I20260809 00:45:05.137681 19324 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:05.137768 19324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:05.139345 19324 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:45:05.140202 19391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.140445 19324 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:05.140556 19324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "49c61f1504ec4974b5af69e56954499d"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "236dd8043456ab8fc0b71ad942d50848"
server_key_iv: "b0d797443be52ca322a9c594632b19e3"
server_key_version: "encryptionkey@0"
I20260809 00:45:05.141072 19324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:05.169565 19324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:05.172289 19324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:05.172449 19324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:05.172722 19324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:05.173327 19402 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:45:05.174939 19324 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:45:05.175007 19324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:45:05.175042 19324 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:45:05.176025 19324 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:45:05.176085 19324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:05.176175 19402 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Bootstrap starting.
I20260809 00:45:05.183880 19324 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:40215
I20260809 00:45:05.184084 19566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:40215 every 8 connection(s)
I20260809 00:45:05.184538 19324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:45:05.185570 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 19324
I20260809 00:45:05.185940 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:41313
--local_ip_for_outbound_sockets=127.30.104.194
--tserver_master_addrs=127.30.104.254:39737
--webserver_port=42865
--webserver_interface=127.30.104.194
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:45:05.187497 19402 log.cc:824] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:05.190891 19402 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:45:05.191259 19402 tablet_bootstrap.cc:492] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Bootstrap complete.
I20260809 00:45:05.192447 19402 ts_tablet_manager.cc:1397] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.011s
I20260809 00:45:05.194335 19402 raft_consensus.cc:348] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.194758 19402 raft_consensus.cc:729] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c61f1504ec4974b5af69e56954499d, State: Initialized, Role: FOLLOWER
I20260809 00:45:05.194976 19402 consensus_queue.cc:260] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.195340 19402 ts_tablet_manager.cc:1428] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:05.201027 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.185920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:33487 (local address 127.30.104.254:39737)
0809 00:45:05.188619 (+ 2699us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.188630 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.188676 (+ 46us) server_negotiation.cc:407] Connection header received
0809 00:45:05.188828 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.188834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.188905 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.189079 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:05.189963 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.191320 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.192943 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.193589 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.193707 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.196782 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:05.196787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:05.196787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:05.196802 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:05.198809 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.199392 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.199395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.199396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.199567 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.199874 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.199876 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.199876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.200186 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:05.200290 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.200657 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.200812 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2632,"thread_start_us":84,"threads_started":1}
I20260809 00:45:05.201064 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.185549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:33487)
0809 00:45:05.187608 (+ 2059us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.187667 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:05.188378 (+ 711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:05.189151 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:05.189156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:05.189522 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:05.189786 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.189793 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.191518 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.191520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.192812 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.192816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.193833 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.193834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.194002 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.194740 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:05.194774 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:05.196640 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:05.198971 (+ 2331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:05.198981 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:05.198983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:05.199266 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:05.199649 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:05.199650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:05.199650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:05.199785 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:05.200272 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:05.200280 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:05.200452 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:45:05.200574 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:45:05.200791 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2003,"thread_start_us":102,"threads_started":1}
I20260809 00:45:05.201766 19567 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39737
I20260809 00:45:05.201875 19567 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:05.202126 19567 heartbeater.cc:511] Master 127.30.104.254:39737 requested a full tablet report, sending...
I20260809 00:45:05.202955 19272 ts_manager.cc:194] Registered new tserver with Master: 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215)
I20260809 00:45:05.204021 19272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:33487
I20260809 00:45:05.205245 19567 heartbeater.cc:503] Master 127.30.104.254:39737 was elected leader, sending a full tablet report...
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236304, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.193@KRBTEST.COM
I20260809 00:45:05.213793 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.204363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:40215 (local address 127.0.0.1:48600)
0809 00:45:05.204591 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.204630 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:05.204695 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:05.204879 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:05.204884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:05.205234 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:05.205351 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.205354 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.206678 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.206680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.207330 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.207332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.208069 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.208071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.208145 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.208773 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:05.208779 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:05.210333 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:05.212583 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:05.212590 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:05.212591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:05.212816 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:05.213055 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:05.213056 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:05.213056 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:05.213115 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:05.213417 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:05.213422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:05.213469 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 00:45:05.213580 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:45:05.213718 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":111,"threads_started":1}
I20260809 00:45:05.213815 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.204428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.30.104.193:40215)
0809 00:45:05.204701 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.204707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.204719 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:45:05.204761 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.204763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.204779 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.204864 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:05.205446 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.206537 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.207485 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.207965 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.208039 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.210493 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:05.210502 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:05.210502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:05.210512 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:05.212480 (+ 1968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.212902 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.212905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.212905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.212985 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.213178 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.213178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.213179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.213347 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:05.213427 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.213621 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.213710 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":116,"threads_started":1}
I20260809 00:45:05.214314 19465 tablet_service.cc:1514] Processing DeleteTablet for tablet 6082296001e74ca886f51b9761ade684 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:04 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:48600
I20260809 00:45:05.214929 19579 tablet_replica.cc:330] stopping tablet replica
I20260809 00:45:05.215049 19579 raft_consensus.cc:2229] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:45:05.215142 19579 raft_consensus.cc:2258] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:05.215525 19579 ts_tablet_manager.cc:1905] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:05.217530 19579 ts_tablet_manager.cc:1918] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:05.217604 19579 log.cc:1196] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/6082296001e74ca886f51b9761ade684
I20260809 00:45:05.217877 19579 ts_tablet_manager.cc:1939] T 6082296001e74ca886f51b9761ade684 P 49c61f1504ec4974b5af69e56954499d: Deleting consensus metadata
I20260809 00:45:05.218446 19245 catalog_manager.cc:4841] TS 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215): tablet 6082296001e74ca886f51b9761ade684 (table test-table [id=ee4f84cdba5d43d080800328893761ca]) successfully deleted
W20260809 00:45:05.280017 19574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:05.280308 19574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:05.280355 19574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:05.281709 19574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:05.281792 19574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:05.281847 19574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:45:05.283828 19574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:41313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=42865
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:05.284158 19574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:05.284441 19574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:05.289850 19591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.289835 19594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.289862 19590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:05.290118 19574 server_base.cc:1034] running on GCE node
I20260809 00:45:05.290457 19574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:05.290745 19574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:05.291945 19574 hybrid_clock.cc:648] HybridClock initialized: now 1786236305291925 us; error 29 us; skew 500 ppm
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:05.294695 19574 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:45:05.295207 19574 webserver.cc:460] Webserver started at http://127.30.104.194:42865/ using document root <none> and password file <none>
I20260809 00:45:05.295513 19574 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:05.295598 19574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:05.297113 19574 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:05.297837 19603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.298081 19574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:05.298182 19574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "3bd155548e1749ac9b690c8b292f9043"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "4771af61b5f69ce119ce4b5435e12594"
server_key_iv: "a0f68754c4259288c4f535066b203959"
server_key_version: "encryptionkey@0"
I20260809 00:45:05.298588 19574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:05.316601 19574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:05.319067 19574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:05.319229 19574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:05.319578 19574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:05.319924 19574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:05.319981 19574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.320025 19574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:05.320055 19574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.326763 19574 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:41313
I20260809 00:45:05.326840 19776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:41313 every 8 connection(s)
I20260809 00:45:05.327271 19574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:05.335677 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 19574
I20260809 00:45:05.335933 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:37267
--local_ip_for_outbound_sockets=127.30.104.195
--tserver_master_addrs=127.30.104.254:39737
--webserver_port=38595
--webserver_interface=127.30.104.195
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:45:05.337579 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.328128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:45641 (local address 127.30.104.254:39737)
0809 00:45:05.328195 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.328197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.328386 (+ 189us) server_negotiation.cc:407] Connection header received
0809 00:45:05.328932 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.328933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.328945 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.329006 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:05.330073 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.330586 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.331744 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.331979 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.332046 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.334405 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:05.334407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:05.334407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:05.334414 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:05.336084 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.336571 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.336572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.336572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.336635 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.336901 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.336902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.336902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.337031 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:05.337103 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.337385 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.337484 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:45:05.337821 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.328110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:45641)
0809 00:45:05.328272 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.328309 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:05.328811 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:05.329080 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:05.329083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:05.329737 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:05.329936 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.329941 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.330711 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.330712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.331636 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.331639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.332067 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.332068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.332181 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.332859 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:05.332869 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:05.334308 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:05.336175 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:05.336183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:05.336186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:05.336482 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:05.336709 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:05.336710 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:05.336710 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:05.336823 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:05.337106 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:05.337113 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:05.337251 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 00:45:05.337362 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 00:45:05.337539 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":97,"threads_started":1}
I20260809 00:45:05.338507 19778 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39737
I20260809 00:45:05.338609 19778 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:05.338840 19778 heartbeater.cc:511] Master 127.30.104.254:39737 requested a full tablet report, sending...
I20260809 00:45:05.339383 19272 ts_manager.cc:194] Registered new tserver with Master: 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313)
I20260809 00:45:05.339941 19272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:45641
W20260809 00:45:05.425344 19782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:05.425567 19782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:05.425591 19782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:05.426673 19782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:05.426708 19782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:05.426734 19782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:45:05.428180 19782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:37267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=38595
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:05.428383 19782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:05.428573 19782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:05.433861 19790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.433876 19794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.433876 19792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:05.434201 19782 server_base.cc:1034] running on GCE node
I20260809 00:45:05.434440 19782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:05.434726 19782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:05.435920 19782 hybrid_clock.cc:648] HybridClock initialized: now 1786236305435887 us; error 44 us; skew 500 ppm
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:05.439002 19782 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:45:05.439688 19782 webserver.cc:460] Webserver started at http://127.30.104.195:38595/ using document root <none> and password file <none>
I20260809 00:45:05.439963 19782 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:05.440032 19782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:05.441609 19782 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:45:05.442296 19807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.442551 19782 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:05.442659 19782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "13c15921d4fd4edf8a07163bbf7cfb5b"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "022ca68bde02b97f286d1374d004a37e"
server_key_iv: "e992a961427f948fcb7275bd0a7c1318"
server_key_version: "encryptionkey@0"
I20260809 00:45:05.443090 19782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:05.456786 19782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:05.459722 19782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:05.459897 19782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:05.460196 19782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:05.460572 19782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:05.460613 19782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.460639 19782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:05.460654 19782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.467531 19782 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:37267
I20260809 00:45:05.467707 19978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:37267 every 8 connection(s)
I20260809 00:45:05.468061 19782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:45:05.473359 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 19782
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:05.480101 19985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.468952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:36637)
0809 00:45:05.469185 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.469241 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:05.469782 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:05.470057 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:05.470061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:05.470686 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:05.470893 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.470898 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.471690 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.471691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.472697 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.472701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.473180 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.473182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.473330 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.474052 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:05.474065 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:05.476220 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:05.478162 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:05.478170 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:05.478173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:05.478507 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:05.478743 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:05.478744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:05.478744 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:05.478894 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:05.479205 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:05.479211 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:05.479492 (+ 281us) client_negotiation.cc:769] Sending connection context
0809 00:45:05.479630 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 00:45:05.479822 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":106,"threads_started":1}
I20260809 00:45:05.480227 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.469064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:36637 (local address 127.30.104.254:39737)
0809 00:45:05.469169 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.469171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.469342 (+ 171us) server_negotiation.cc:407] Connection header received
0809 00:45:05.469902 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.469904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.469918 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.469974 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:05.471018 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.471592 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.472842 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.473084 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.473174 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.476325 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:05.476326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:05.476326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:05.476334 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:05.478064 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.478597 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.478598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.478598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.478656 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.478970 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.478972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.478972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.479126 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:05.479202 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.479910 (+ 708us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.480076 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 00:45:05.481011 19979 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39737
I20260809 00:45:05.481141 19979 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:05.481359 19979 heartbeater.cc:511] Master 127.30.104.254:39737 requested a full tablet report, sending...
I20260809 00:45:05.481774 19272 ts_manager.cc:194] Registered new tserver with Master: 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267)
I20260809 00:45:05.482079 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.473878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38184 (local address 127.30.104.254:39737)
0809 00:45:05.474060 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.474063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.474119 (+ 56us) server_negotiation.cc:407] Connection header received
0809 00:45:05.474210 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.474210 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.474220 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.474283 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:05.475146 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.475964 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.476913 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.477142 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.477226 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.478672 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:05.478673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:05.478673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:05.478681 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:05.480800 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.481239 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.481241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.481241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.481305 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:05.481566 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:05.481567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:05.481568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:05.481713 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:05.481850 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.481938 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.482016 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":78,"threads_started":1}
I20260809 00:45:05.482322 19272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:36637
I20260809 00:45:05.483158 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:45:05.485924 19991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.483545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38186 (local address 127.30.104.254:39737)
0809 00:45:05.483602 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.483604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.483668 (+ 64us) server_negotiation.cc:407] Connection header received
0809 00:45:05.483721 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.483721 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.483752 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.483818 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:45:05.483914 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.484415 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.485169 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.485385 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.485452 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.485629 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:45:05.485737 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:45:05.485799 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.485858 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.485877 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:45:05.486248 19270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38186:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:45:05.488006 19270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:45:05.491778 19465 tablet_service.cc:1467] Processing CreateTablet for tablet 3df91659bd1342d5810ccb33ea38bfa3 (DEFAULT_TABLE table=test-table [id=a4a89c26d66f4dc2bc3d6d69303b7684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:05.492024 19465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df91659bd1342d5810ccb33ea38bfa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:05.493690 19402 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Bootstrap starting.
I20260809 00:45:05.494014 19402 tablet_bootstrap.cc:654] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:05.494832 19402 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: No bootstrap required, opened a new log
I20260809 00:45:05.494894 19402 ts_tablet_manager.cc:1397] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:05.495049 19402 raft_consensus.cc:348] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.495129 19402 raft_consensus.cc:374] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:05.495177 19402 raft_consensus.cc:729] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c61f1504ec4974b5af69e56954499d, State: Initialized, Role: FOLLOWER
I20260809 00:45:05.495230 19402 consensus_queue.cc:260] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.495322 19402 ts_tablet_manager.cc:1428] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 00:45:05.495997 19577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.491311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:41313 (local address 127.0.0.1:46684)
0809 00:45:05.491462 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.491474 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:05.491521 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:05.491980 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:05.491981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:05.491984 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:05.492037 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.492039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.493295 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.493296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.494660 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.494663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.495696 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.495697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.495774 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.495797 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 00:45:05.495867 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:45:05.495922 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260809 00:45:05.496008 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.491554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.30.104.194:41313)
0809 00:45:05.491764 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.491769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.491782 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:45:05.491828 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.491830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.491847 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.492115 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:05.492204 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.493181 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.494776 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.495580 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.495695 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.495758 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.495889 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.495931 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":83,"threads_started":1}
I20260809 00:45:05.496790 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.491420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:37267 (local address 127.0.0.1:44040)
0809 00:45:05.491621 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.491631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:05.491676 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:05.492095 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:05.492096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:05.492098 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:05.492132 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.492134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.493442 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.493443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.494660 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.494663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.496550 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.496551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.496617 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.496622 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:45:05.496679 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 00:45:05.496720 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":8,"thread_start_us":131,"threads_started":1}
I20260809 00:45:05.496804 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.491691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.30.104.195:37267)
0809 00:45:05.491896 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.491900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.491912 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:45:05.491957 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.491959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.491976 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.492095 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:05.492204 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.493324 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.495515 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.496458 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.496538 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.496591 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.496700 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.496735 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
I20260809 00:45:05.496920 19669 tablet_service.cc:1467] Processing CreateTablet for tablet 3df91659bd1342d5810ccb33ea38bfa3 (DEFAULT_TABLE table=test-table [id=a4a89c26d66f4dc2bc3d6d69303b7684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:05.497318 19669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df91659bd1342d5810ccb33ea38bfa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:05.497656 19880 tablet_service.cc:1467] Processing CreateTablet for tablet 3df91659bd1342d5810ccb33ea38bfa3 (DEFAULT_TABLE table=test-table [id=a4a89c26d66f4dc2bc3d6d69303b7684]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:05.497910 19880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3df91659bd1342d5810ccb33ea38bfa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:05.500186 19998 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Bootstrap starting.
I20260809 00:45:05.500960 19998 tablet_bootstrap.cc:654] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:05.500941 20000 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: Bootstrap starting.
I20260809 00:45:05.501523 19998 log.cc:824] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:05.501667 20000 tablet_bootstrap.cc:654] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:05.502228 20000 log.cc:824] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:05.502527 19998 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: No bootstrap required, opened a new log
I20260809 00:45:05.502619 19998 ts_tablet_manager.cc:1397] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:45:05.502939 20000 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: No bootstrap required, opened a new log
I20260809 00:45:05.502997 20000 ts_tablet_manager.cc:1397] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:45:05.504501 19998 raft_consensus.cc:348] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.504582 20000 raft_consensus.cc:348] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.504671 19998 raft_consensus.cc:374] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:05.504698 20000 raft_consensus.cc:374] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:05.504726 19998 raft_consensus.cc:729] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd155548e1749ac9b690c8b292f9043, State: Initialized, Role: FOLLOWER
I20260809 00:45:05.504750 20000 raft_consensus.cc:729] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c15921d4fd4edf8a07163bbf7cfb5b, State: Initialized, Role: FOLLOWER
I20260809 00:45:05.504853 19998 consensus_queue.cc:260] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.504904 20000 consensus_queue.cc:260] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.505193 19778 heartbeater.cc:503] Master 127.30.104.254:39737 was elected leader, sending a full tablet report...
I20260809 00:45:05.505227 20000 ts_tablet_manager.cc:1428] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:45:05.505276 19979 heartbeater.cc:503] Master 127.30.104.254:39737 was elected leader, sending a full tablet report...
I20260809 00:45:05.505484 19998 ts_tablet_manager.cc:1428] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260809 00:45:05.578015 19779 tablet.cc:2369] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:45:05.582324 20003 raft_consensus.cc:482] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:05.582453 20003 raft_consensus.cc:504] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.582825 20003 leader_election.cc:288] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267), 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215)
I20260809 00:45:05.586086 19996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.583182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:44311 (local address 127.30.104.195:37267)
0809 00:45:05.583244 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.583247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.583260 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:45:05.583293 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.583294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.583315 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.583401 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:05.583629 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.584126 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.585090 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.585672 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.585756 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.585766 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.585951 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.585996 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:45:05.586072 19780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.583053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:37267 (local address 127.30.104.194:44311)
0809 00:45:05.583129 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.583143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:05.583216 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:05.583421 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:05.583423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:05.583425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:05.583539 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.583542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.584238 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.584239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.584986 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.584988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.585753 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.585754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.585849 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.585852 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:45:05.585926 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:45:05.585979 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 00:45:05.586162 19578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.583206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:58077 (local address 127.30.104.193:40215)
0809 00:45:05.583268 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.583270 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.583398 (+ 128us) server_negotiation.cc:407] Connection header received
0809 00:45:05.583496 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.583497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.583523 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.583615 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:05.583790 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.584300 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.585281 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.585836 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.585894 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.585993 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.586049 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.586090 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:45:05.586184 20005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.583053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:40215 (local address 127.30.104.194:58077)
0809 00:45:05.583329 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.583337 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:05.583402 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:05.583629 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:05.583630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:05.583631 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:05.583714 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.583716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.584389 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.584390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.585172 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:05.585175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.585917 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.585917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:05.585983 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.585986 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:45:05.586030 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:45:05.586061 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":92,"thread_start_us":113,"threads_started":1}
I20260809 00:45:05.586506 19908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df91659bd1342d5810ccb33ea38bfa3" candidate_uuid: "3bd155548e1749ac9b690c8b292f9043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" is_pre_election: true
I20260809 00:45:05.586688 19908 raft_consensus.cc:2454] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd155548e1749ac9b690c8b292f9043 in term 0.
I20260809 00:45:05.586683 19498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df91659bd1342d5810ccb33ea38bfa3" candidate_uuid: "3bd155548e1749ac9b690c8b292f9043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c61f1504ec4974b5af69e56954499d" is_pre_election: true
I20260809 00:45:05.586854 19498 raft_consensus.cc:2454] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bd155548e1749ac9b690c8b292f9043 in term 0.
I20260809 00:45:05.586947 19608 leader_election.cc:302] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043; no voters:
I20260809 00:45:05.587085 20003 raft_consensus.cc:2781] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:45:05.587143 20003 raft_consensus.cc:482] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:45:05.587173 20003 raft_consensus.cc:3037] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:05.588019 20003 raft_consensus.cc:504] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.588181 20003 leader_election.cc:288] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [CANDIDATE]: Term 1 election: Requested vote from peers 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267), 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215)
I20260809 00:45:05.588336 19908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df91659bd1342d5810ccb33ea38bfa3" candidate_uuid: "3bd155548e1749ac9b690c8b292f9043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b"
I20260809 00:45:05.588392 19498 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3df91659bd1342d5810ccb33ea38bfa3" candidate_uuid: "3bd155548e1749ac9b690c8b292f9043" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c61f1504ec4974b5af69e56954499d"
I20260809 00:45:05.588421 19908 raft_consensus.cc:3037] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:05.588500 19498 raft_consensus.cc:3037] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:05.589187 19498 raft_consensus.cc:2454] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd155548e1749ac9b690c8b292f9043 in term 1.
I20260809 00:45:05.589195 19908 raft_consensus.cc:2454] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bd155548e1749ac9b690c8b292f9043 in term 1.
I20260809 00:45:05.589413 19608 leader_election.cc:302] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043; no voters:
I20260809 00:45:05.589542 20003 raft_consensus.cc:2781] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:45:05.589735 20003 raft_consensus.cc:686] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd155548e1749ac9b690c8b292f9043, State: Running, Role: LEADER
I20260809 00:45:05.589815 20003 consensus_queue.cc:237] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:05.590776 19272 catalog_manager.cc:5495] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194). New cstate: current_term: 1 leader_uuid: "3bd155548e1749ac9b690c8b292f9043" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } health_report { overall_health: UNKNOWN } } }
I20260809 00:45:05.614149 19994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.611322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.30.104.194:41313)
0809 00:45:05.611465 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:05.611468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:05.611478 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:45:05.611522 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:05.611523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:05.611554 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:05.611623 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:45:05.611894 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.612370 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.613179 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:05.613372 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:05.613473 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:05.613719 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:45:05.613842 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:45:05.613931 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:05.614039 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 00:45:05.614062 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 00:45:05.616717 19498 raft_consensus.cc:1261] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Refusing update from remote peer 3bd155548e1749ac9b690c8b292f9043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:45:05.616715 19908 raft_consensus.cc:1261] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 1 FOLLOWER]: Refusing update from remote peer 3bd155548e1749ac9b690c8b292f9043: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:45:05.617156 20006 consensus_queue.cc:1035] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:45:05.617326 20003 consensus_queue.cc:1035] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:45:05.620461 20010 mvcc.cc:205] Tried to move back new op lower bound from 7316423907802251264 to 7316423907696443392. Current Snapshot: MvccSnapshot[applied={T|T < 7316423907802251264}]
I20260809 00:45:05.620862 20013 mvcc.cc:205] Tried to move back new op lower bound from 7316423907802251264 to 7316423907696443392. Current Snapshot: MvccSnapshot[applied={T|T < 7316423907802251264}]
I20260809 00:45:05.621620 20012 mvcc.cc:205] Tried to move back new op lower bound from 7316423907802251264 to 7316423907696443392. Current Snapshot: MvccSnapshot[applied={T|T < 7316423907802251264}]
I20260809 00:45:05.623816 19272 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:45:05.623931 19272 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38186:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:45:05.625044 19272 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Sending DeleteTablet for 3 replicas of tablet 3df91659bd1342d5810ccb33ea38bfa3
I20260809 00:45:05.625444 19465 tablet_service.cc:1514] Processing DeleteTablet for tablet 3df91659bd1342d5810ccb33ea38bfa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:05 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:48600
I20260809 00:45:05.625806 19880 tablet_service.cc:1514] Processing DeleteTablet for tablet 3df91659bd1342d5810ccb33ea38bfa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:05 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:44040
I20260809 00:45:05.626058 19669 tablet_service.cc:1514] Processing DeleteTablet for tablet 3df91659bd1342d5810ccb33ea38bfa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:05 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:46684
I20260809 00:45:05.626107 20022 tablet_replica.cc:330] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: stopping tablet replica
I20260809 00:45:05.626235 20022 raft_consensus.cc:2229] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:45:05.626332 19579 tablet_replica.cc:330] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: stopping tablet replica
I20260809 00:45:05.626405 19579 raft_consensus.cc:2229] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:45:05.626415 20022 raft_consensus.cc:2258] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:05.626463 19579 raft_consensus.cc:2258] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:05.626434 20029 tablet_replica.cc:330] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: stopping tablet replica
I20260809 00:45:05.626693 20029 raft_consensus.cc:2229] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:45:05.626842 20029 raft_consensus.cc:2258] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:05.627027 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 19324
I20260809 00:45:05.629451 20022 ts_tablet_manager.cc:1905] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:05.632203 20022 ts_tablet_manager.cc:1918] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:05.632325 20022 log.cc:1196] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/3df91659bd1342d5810ccb33ea38bfa3
I20260809 00:45:05.632819 20022 ts_tablet_manager.cc:1939] T 3df91659bd1342d5810ccb33ea38bfa3 P 13c15921d4fd4edf8a07163bbf7cfb5b: Deleting consensus metadata
I20260809 00:45:05.633764 19246 catalog_manager.cc:4841] TS 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267): tablet 3df91659bd1342d5810ccb33ea38bfa3 (table test-table [id=a4a89c26d66f4dc2bc3d6d69303b7684]) successfully deleted
I20260809 00:45:05.637207 20029 ts_tablet_manager.cc:1905] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:05.639605 20029 ts_tablet_manager.cc:1918] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:05.639686 20029 log.cc:1196] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/3df91659bd1342d5810ccb33ea38bfa3
I20260809 00:45:05.639956 20029 ts_tablet_manager.cc:1939] T 3df91659bd1342d5810ccb33ea38bfa3 P 3bd155548e1749ac9b690c8b292f9043: Deleting consensus metadata
I20260809 00:45:05.640801 19246 catalog_manager.cc:4841] TS 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313): tablet 3df91659bd1342d5810ccb33ea38bfa3 (table test-table [id=a4a89c26d66f4dc2bc3d6d69303b7684]) successfully deleted
W20260809 00:45:05.647255 19245 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:40215 (error 108)
I20260809 00:45:05.647624 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 19574
I20260809 00:45:05.649979 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:05.649751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:40215 (local address 127.0.0.1:48602)
0809 00:45:05.649823 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:05.649879 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:40215: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 00:45:05.652842 19245 catalog_manager.cc:4584] TS 49c61f1504ec4974b5af69e56954499d (127.30.104.193:40215): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3df91659bd1342d5810ccb33ea38bfa3: Network error: Client connection negotiation failed: client connection to 127.30.104.193:40215: connect: Connection refused (error 111)
I20260809 00:45:05.656838 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 19782
I20260809 00:45:05.673522 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 19218
I20260809 00:45:05.686080 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39737
--webserver_interface=127.30.104.254
--webserver_port=43463
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:05.772117 20034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:05.772338 20034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:05.772370 20034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:05.773497 20034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:45:05.773532 20034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:05.773546 20034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:05.773558 20034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:45:05.773571 20034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:45:05.775007 20034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:39737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=43463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:05.775213 20034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:05.775398 20034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:05.780823 20048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.780927 20045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.780823 20043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:05.781133 20034 server_base.cc:1034] running on GCE node
I20260809 00:45:05.781399 20034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:05.781786 20034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:05.782967 20034 hybrid_clock.cc:648] HybridClock initialized: now 1786236305782942 us; error 40 us; skew 500 ppm
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:05.785676 20034 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:45:05.786157 20034 webserver.cc:460] Webserver started at http://127.30.104.254:43463/ using document root <none> and password file <none>
I20260809 00:45:05.786350 20034 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:05.786391 20034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:05.788174 20034 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:45:05.788831 20058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.789040 20034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:05.789166 20034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "b01c68c633bd9b3c3e0b9b91158d40bf"
server_key_iv: "ea562193dd93e4829bff8882f40c2d94"
server_key_version: "encryptionkey@0"
I20260809 00:45:05.789574 20034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:05.812274 20034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:05.814792 20034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:05.814951 20034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:05.819320 20034 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:39737
I20260809 00:45:05.819357 20142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:39737 every 8 connection(s)
I20260809 00:45:05.819825 20034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:45:05.821607 20143 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:45:05.822387 20143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Bootstrap starting.
I20260809 00:45:05.822559 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 20034
I20260809 00:45:05.822890 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:40215
--local_ip_for_outbound_sockets=127.30.104.193
--tserver_master_addrs=127.30.104.254:39737
--webserver_port=39947
--webserver_interface=127.30.104.193
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:45:05.833783 20143 log.cc:824] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:05.837568 20143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:45:05.837961 20143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7: Bootstrap complete.
I20260809 00:45:05.841413 20143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:05.841732 20143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f38e31f7fbe4ba7a51a77e295a9e9d7, State: Initialized, Role: FOLLOWER
I20260809 00:45:05.841892 20143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:05.841958 20143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:45:05.842006 20143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:45:05.842053 20143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:45:05.843401 20143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:05.843547 20143 leader_election.cc:302] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f38e31f7fbe4ba7a51a77e295a9e9d7; no voters:
I20260809 00:45:05.843722 20143 leader_election.cc:288] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 00:45:05.843873 20147 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 00:45:05.844020 20143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:45:05.844045 20147 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [term 3 LEADER]: Becoming Leader. State: Replica: 2f38e31f7fbe4ba7a51a77e295a9e9d7, State: Running, Role: LEADER
I20260809 00:45:05.844158 20147 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } }
I20260809 00:45:05.844434 20148 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } } }
I20260809 00:45:05.844525 20148 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:05.844933 20149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f38e31f7fbe4ba7a51a77e295a9e9d7. Latest consensus state: current_term: 3 leader_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f38e31f7fbe4ba7a51a77e295a9e9d7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39737 } } }
I20260809 00:45:05.845117 20149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f38e31f7fbe4ba7a51a77e295a9e9d7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:05.845289 20156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:45:05.846191 20156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:45:05.846316 20156 catalog_manager.cc:1259] Loaded cluster ID: 645ee976489d46d290837a69059f191a
I20260809 00:45:05.846611 20156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 00:45:05.846891 20156 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb8ec95e9ad google::LogMessage::Fail() at ??:0
@ 0x7fb8ec96188a google::LogMessage::SendToLog() at ??:0
@ 0x7fb8ec95e4de google::LogMessage::Flush() at ??:0
@ 0x7fb8ec95fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb8f487f0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fb8f48888b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb8ee82c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb8ee821954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb8ef6856db start_thread at ??:0
@ 0x7fb8eb45c61f clone at ??:0
W20260809 00:45:05.959707 20145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:05.960044 20145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:05.960091 20145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:05.962077 20145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:05.962154 20145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:05.962203 20145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:45:05.964375 20145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:40215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=39947
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:05.964694 20145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:05.964960 20145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:05.970948 20196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.970960 20198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:05.970960 20195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:05.971231 20145 server_base.cc:1034] running on GCE node
I20260809 00:45:05.971560 20145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:05.971850 20145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:05.973057 20145 hybrid_clock.cc:648] HybridClock initialized: now 1786236305973018 us; error 56 us; skew 500 ppm
Aug 09 00:45:05 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:05.976079 20145 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:45:05.976665 20145 webserver.cc:460] Webserver started at http://127.30.104.193:39947/ using document root <none> and password file <none>
I20260809 00:45:05.976934 20145 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:05.977016 20145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:05.978667 20145 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:05.979550 20212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:05.979748 20145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:05.979878 20145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "49c61f1504ec4974b5af69e56954499d"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "236dd8043456ab8fc0b71ad942d50848"
server_key_iv: "b0d797443be52ca322a9c594632b19e3"
server_key_version: "encryptionkey@0"
I20260809 00:45:05.980293 20145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:05.997033 20145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:05.999856 20145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:06.000034 20145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:06.000336 20145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:06.000875 20223 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:45:06.003012 20145 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:45:06.003103 20145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:45:06.003160 20145 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:45:06.004020 20145 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:45:06.004091 20145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:45:06.004153 20223 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Bootstrap starting.
I20260809 00:45:06.012363 20145 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:40215
I20260809 00:45:06.012912 20145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:45:06.013370 20379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:40215 every 8 connection(s)
I20260809 00:45:06.015254 20223 log.cc:824] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:06.018126 20223 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:45:06.018481 20223 tablet_bootstrap.cc:492] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Bootstrap complete.
I20260809 00:45:06.019757 20223 ts_tablet_manager.cc:1397] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.013s
I20260809 00:45:06.021257 20223 raft_consensus.cc:348] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:06.021641 20223 raft_consensus.cc:729] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c61f1504ec4974b5af69e56954499d, State: Initialized, Role: FOLLOWER
I20260809 00:45:06.021790 20223 consensus_queue.cc:260] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:06.022080 20223 ts_tablet_manager.cc:1428] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:45:06.023095 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 20145
I20260809 00:45:06.023378 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:41313
--local_ip_for_outbound_sockets=127.30.104.194
--tserver_master_addrs=127.30.104.254:39737
--webserver_port=42865
--webserver_interface=127.30.104.194
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:06.093549 20382 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:45:06.013700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:33111)
0809 00:45:06.013880 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.013926 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:06.014468 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:06.093327 (+ 78859us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":110,"threads_started":1}
W20260809 00:45:06.093848 20380 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:39737 (0 consecutive failures): Network error: Failed to ping master at 127.30.104.254:39737: Client connection negotiation failed: client connection to 127.30.104.254:39737: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 00:45:06.094233 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:06.094096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:55009)
0809 00:45:06.094149 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.094200 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:45:06.094640 20382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:06.094530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:45329)
0809 00:45:06.094584 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.094617 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 00:45:06.094823 20380 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:45:06.123724 20388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:06.123958 20388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:06.123983 20388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:06.125252 20388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:06.125300 20388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:06.125327 20388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:45:06.126844 20388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:41313
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=42865
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:06.127127 20388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:06.127391 20388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:06.132994 20403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:06.133010 20400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:06.133008 20398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:06.133373 20388 server_base.cc:1034] running on GCE node
I20260809 00:45:06.133592 20388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:06.133860 20388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:06.135092 20388 hybrid_clock.cc:648] HybridClock initialized: now 1786236306135082 us; error 31 us; skew 500 ppm
Aug 09 00:45:06 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236306, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:06.137948 20388 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:45:06.138414 20388 webserver.cc:460] Webserver started at http://127.30.104.194:42865/ using document root <none> and password file <none>
I20260809 00:45:06.138662 20388 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:06.138734 20388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:06.140247 20388 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:45:06.140950 20413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:06.141273 20388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:06.141391 20388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "3bd155548e1749ac9b690c8b292f9043"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "4771af61b5f69ce119ce4b5435e12594"
server_key_iv: "a0f68754c4259288c4f535066b203959"
server_key_version: "encryptionkey@0"
I20260809 00:45:06.141781 20388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:06.150792 20388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:06.152940 20388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:06.153108 20388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:06.153362 20388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:06.153714 20388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:06.153772 20388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:06.153816 20388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:06.153846 20388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:06.160238 20388 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:41313
I20260809 00:45:06.160298 20581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:41313 every 8 connection(s)
I20260809 00:45:06.160698 20388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:45:06.162012 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:06.161434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:57335)
0809 00:45:06.161593 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.161672 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
W20260809 00:45:06.162317 20582 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:39737 (0 consecutive failures): Network error: Failed to ping master at 127.30.104.254:39737: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
I20260809 00:45:06.162652 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:06.162553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:37819)
0809 00:45:06.162603 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.162632 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 00:45:06.162887 20584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:06.162837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:40943)
0809 00:45:06.162857 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.162875 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 00:45:06.162997 20582 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:45:06.170902 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 20388
I20260809 00:45:06.171136 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:37267
--local_ip_for_outbound_sockets=127.30.104.195
--tserver_master_addrs=127.30.104.254:39737
--webserver_port=38595
--webserver_interface=127.30.104.195
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:41821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:06.256747 20585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:06.257045 20585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:06.257088 20585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:06.258415 20585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:06.258493 20585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:06.258540 20585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:45:06.260106 20585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:41821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:37267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=38595
--tserver_master_addrs=127.30.104.254:39737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:06.260385 20585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:06.260648 20585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:06.266094 20591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:06.266100 20596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:06.266100 20590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:06.266507 20585 server_base.cc:1034] running on GCE node
I20260809 00:45:06.266759 20585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:06.267055 20585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:06.268249 20585 hybrid_clock.cc:648] HybridClock initialized: now 1786236306268236 us; error 32 us; skew 500 ppm
Aug 09 00:45:06 dist-test-slave-c12x krb5kdc[18377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236306, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:06.271065 20585 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:45:06.271634 20585 webserver.cc:460] Webserver started at http://127.30.104.195:38595/ using document root <none> and password file <none>
I20260809 00:45:06.271905 20585 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:06.271981 20585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:06.273401 20585 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:06.274260 20606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:06.274540 20585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:06.274658 20585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "13c15921d4fd4edf8a07163bbf7cfb5b"
format_stamp: "Formatted at 2026-08-09 00:45:04 on dist-test-slave-c12x"
server_key: "022ca68bde02b97f286d1374d004a37e"
server_key_iv: "e992a961427f948fcb7275bd0a7c1318"
server_key_version: "encryptionkey@0"
I20260809 00:45:06.275055 20585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:06.285418 20585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:06.287846 20585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:06.288025 20585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:06.288292 20585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:06.288618 20585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:06.288672 20585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:06.288707 20585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:06.288738 20585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:06.294816 20585 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:37267
I20260809 00:45:06.294864 20790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:37267 every 8 connection(s)
I20260809 00:45:06.295226 20585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:45:06.296589 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:06.296107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:42575)
0809 00:45:06.296320 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.296426 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":84,"threads_started":1}
W20260809 00:45:06.297101 20794 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:39737 (0 consecutive failures): Network error: Failed to ping master at 127.30.104.254:39737: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
I20260809 00:45:06.297431 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:06.297321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:34009)
0809 00:45:06.297378 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.297410 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:45:06.297811 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 20585
I20260809 00:45:06.297891 20799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:06.297760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:39135)
0809 00:45:06.297830 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:06.297870 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 00:45:06.298027 20794 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:45:07.095643 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:07.095329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:53693)
0809 00:45:07.095525 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:07.095570 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":85,"threads_started":1}
I20260809 00:45:07.163763 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:07.163540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:60121)
0809 00:45:07.163665 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:07.163702 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260809 00:45:07.298715 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:07.298473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:35913)
0809 00:45:07.298612 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:07.298658 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":65,"threads_started":1}
I20260809 00:45:07.874058 20804 raft_consensus.cc:482] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:07.874181 20804 raft_consensus.cc:504] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:07.874487 20804 leader_election.cc:288] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313), 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267)
Aug 09 00:45:07 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.194@KRBTEST.COM
Aug 09 00:45:07 dist-test-slave-c12x krb5kdc[18377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236305, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.195@KRBTEST.COM
I20260809 00:45:07.889322 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:07.874698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:41313 (local address 127.30.104.193:38439)
0809 00:45:07.874806 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:07.874818 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:07.874878 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:07.875838 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:07.875842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:07.876404 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:07.876697 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:07.876706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:07.878556 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:07.878556 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:07.879921 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:07.879926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:07.880835 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:07.880837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:07.880962 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:07.881542 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:07.881574 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:07.883743 (+ 2169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:07.887321 (+ 3578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:07.887330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:07.887332 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:07.887802 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:07.888097 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:07.888098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:07.888098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:07.888222 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:07.888690 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:07.888697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:07.888854 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 00:45:07.888973 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 00:45:07.889214 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":68,"thread_start_us":60,"threads_started":1}
I20260809 00:45:07.889595 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:07.874792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:38439 (local address 127.30.104.194:41313)
0809 00:45:07.874970 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:07.874983 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:07.875010 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:45:07.875592 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:07.875595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:07.875640 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:07.875765 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:07.876843 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:07.878401 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:07.880050 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:07.880719 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:07.880845 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:07.883872 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:07.883877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:07.883878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:07.883890 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:07.887208 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:07.887899 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:07.887902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:07.887903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:07.888021 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:07.888287 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:07.888288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:07.888288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:07.888611 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:07.888698 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:07.889299 (+ 601us) server_negotiation.cc:299] Negotiation successful
0809 00:45:07.889451 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":72,"threads_started":1}
I20260809 00:45:07.889974 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:07.874873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:37267 (local address 127.30.104.193:50397)
0809 00:45:07.874978 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:07.874988 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:07.875030 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:07.875866 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:07.875868 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:07.876404 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:07.876696 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:07.876705 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:07.878509 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:07.878512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:07.879922 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:07.879926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:07.881167 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:07.881168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:07.881207 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:07.881711 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:07.881716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:07.884382 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:07.887360 (+ 2978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:07.887361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:07.887361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:07.888087 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:07.888448 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:07.888449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:07.888450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:07.888515 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:07.888987 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:07.888988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:07.889048 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 00:45:07.889143 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 00:45:07.889855 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":659,"thread_start_us":61,"threads_started":1}
I20260809 00:45:07.890002 20811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:07.874986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:50397 (local address 127.30.104.195:37267)
0809 00:45:07.875138 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:07.875147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:07.875166 (+ 19us) server_negotiation.cc:407] Connection header received
0809 00:45:07.875659 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:07.875662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:07.875704 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:07.875861 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:07.876838 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:07.878401 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:07.880037 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:07.880727 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:07.880845 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:07.884514 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:07.884519 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:07.884519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:07.884536 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:07.887242 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:07.888197 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:07.888200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:07.888201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:07.888357 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:07.888575 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:07.888576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:07.888577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:07.888898 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:07.888992 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:07.889240 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 00:45:07.889376 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":62,"threads_started":1}
W20260809 00:45:07.890974 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
W20260809 00:45:07.891067 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
I20260809 00:45:07.891103 20218 leader_election.cc:302] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c61f1504ec4974b5af69e56954499d; no voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043
I20260809 00:45:07.891203 20804 raft_consensus.cc:2726] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:45:08.096453 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:08.096274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:42979)
0809 00:45:08.096366 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:08.096410 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 00:45:08.164672 20813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:08.164368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:55141)
0809 00:45:08.164556 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:08.164613 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":83,"threads_started":1}
I20260809 00:45:08.299669 20815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:08.299323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:57385)
0809 00:45:08.299561 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:08.299616 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":72,"threads_started":1}
I20260809 00:45:09.097332 20817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:09.097071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:45237)
0809 00:45:09.097241 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:09.097281 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":77,"threads_started":1}
I20260809 00:45:09.165504 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:09.165256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:41097)
0809 00:45:09.165409 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:09.165455 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260809 00:45:09.300473 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:09.300215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:46537)
0809 00:45:09.300371 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:09.300421 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
I20260809 00:45:09.955361 20820 raft_consensus.cc:482] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:09.955546 20820 raft_consensus.cc:504] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:09.955762 20820 leader_election.cc:288] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313), 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267)
W20260809 00:45:09.956434 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
W20260809 00:45:09.956545 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
I20260809 00:45:09.956612 20218 leader_election.cc:302] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c61f1504ec4974b5af69e56954499d; no voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043
I20260809 00:45:09.956720 20820 raft_consensus.cc:2726] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:45:10.098464 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:10.098073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:58691)
0809 00:45:10.098309 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:10.098388 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260809 00:45:10.166401 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:10.166138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:46071)
0809 00:45:10.166290 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:10.166334 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260809 00:45:10.301374 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:10.301105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:51733)
0809 00:45:10.301262 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:10.301309 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":70,"threads_started":1}
I20260809 00:45:11.099326 20824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:11.099100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:35893)
0809 00:45:11.099235 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:11.099278 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260809 00:45:11.167239 20825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:11.166992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:48459)
0809 00:45:11.167146 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:11.167188 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":78,"threads_started":1}
I20260809 00:45:11.302241 20826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:11.301988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:40797)
0809 00:45:11.302141 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:11.302189 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":67,"threads_started":1}
I20260809 00:45:11.643599 20827 raft_consensus.cc:482] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:11.643712 20827 raft_consensus.cc:504] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:11.643930 20827 leader_election.cc:288] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313), 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267)
W20260809 00:45:11.644451 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
W20260809 00:45:11.644527 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
I20260809 00:45:11.644544 20218 leader_election.cc:302] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c61f1504ec4974b5af69e56954499d; no voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043
I20260809 00:45:11.644729 20827 raft_consensus.cc:2726] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:45:12.100405 20828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:12.100012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:51023)
0809 00:45:12.100280 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:12.100351 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":138,"threads_started":1}
I20260809 00:45:12.169310 20830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:12.168735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:56987)
0809 00:45:12.168900 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:12.169024 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260809 00:45:12.303113 20831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:12.302852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:49895)
0809 00:45:12.303016 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:12.303063 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":73,"threads_started":1}
I20260809 00:45:13.101285 20832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:13.101051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:49461)
0809 00:45:13.101191 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:13.101234 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260809 00:45:13.170433 20835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:13.170128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:52315)
0809 00:45:13.170291 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:13.170379 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":71,"threads_started":1}
I20260809 00:45:13.304039 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:13.303775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:36509)
0809 00:45:13.303931 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:13.303973 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260809 00:45:13.599488 20838 raft_consensus.cc:482] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:13.599642 20838 raft_consensus.cc:504] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:13.599857 20838 leader_election.cc:288] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313), 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267)
W20260809 00:45:13.600279 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
W20260809 00:45:13.600339 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
I20260809 00:45:13.600356 20218 leader_election.cc:302] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c61f1504ec4974b5af69e56954499d; no voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043
I20260809 00:45:13.600437 20838 raft_consensus.cc:2726] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:45:14.102029 20839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:14.101790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:37121)
0809 00:45:14.101944 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:14.101981 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":62,"threads_started":1}
I20260809 00:45:14.171241 20840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:14.171025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:57835)
0809 00:45:14.171154 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:14.171196 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260809 00:45:14.304847 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:14.304591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:41581)
0809 00:45:14.304753 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:14.304795 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":72,"threads_started":1}
I20260809 00:45:15.102908 20844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:15.102639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:39061)
0809 00:45:15.102804 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:15.102855 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":69,"threads_started":1}
I20260809 00:45:15.172109 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:15.171834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:49341)
0809 00:45:15.172014 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:15.172061 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":85,"threads_started":1}
I20260809 00:45:15.305809 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:15.305500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:57811)
0809 00:45:15.305713 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:15.305761 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":113,"threads_started":1}
I20260809 00:45:16.103847 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:16.103584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:40755)
0809 00:45:16.103746 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:16.103797 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":65,"threads_started":1}
I20260809 00:45:16.172942 20849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:16.172675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:38933)
0809 00:45:16.172852 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:16.172895 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":95,"threads_started":1}
I20260809 00:45:16.306694 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:16.306434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:43447)
0809 00:45:16.306611 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:16.306647 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
I20260809 00:45:16.405007 20852 raft_consensus.cc:482] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:16.405113 20852 raft_consensus.cc:504] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bd155548e1749ac9b690c8b292f9043" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41313 } } peers { permanent_uuid: "13c15921d4fd4edf8a07163bbf7cfb5b" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 37267 } } peers { permanent_uuid: "49c61f1504ec4974b5af69e56954499d" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40215 } }
I20260809 00:45:16.405349 20852 leader_election.cc:288] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313), 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267)
W20260809 00:45:16.405834 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 13c15921d4fd4edf8a07163bbf7cfb5b (127.30.104.195:37267): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
W20260809 00:45:16.405923 20218 leader_election.cc:341] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bd155548e1749ac9b690c8b292f9043 (127.30.104.194:41313): Not found: Tablet not found: 3df91659bd1342d5810ccb33ea38bfa3
I20260809 00:45:16.405962 20218 leader_election.cc:302] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 49c61f1504ec4974b5af69e56954499d; no voters: 13c15921d4fd4edf8a07163bbf7cfb5b, 3bd155548e1749ac9b690c8b292f9043
I20260809 00:45:16.406046 20852 raft_consensus.cc:2726] T 3df91659bd1342d5810ccb33ea38bfa3 P 49c61f1504ec4974b5af69e56954499d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:45:17.104866 20855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:17.104579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:54305)
0809 00:45:17.104766 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:17.104817 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260809 00:45:17.173859 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:17.173600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:53561)
0809 00:45:17.173762 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:17.173808 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260809 00:45:17.307704 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:17.307325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:53251)
0809 00:45:17.307598 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:17.307647 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":206,"threads_started":1}
I20260809 00:45:18.105836 20859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:18.105592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:36085)
0809 00:45:18.105742 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:18.105788 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260809 00:45:18.174710 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:18.174470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:52745)
0809 00:45:18.174613 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:18.174652 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260809 00:45:18.308622 20862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:18.308340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:41405)
0809 00:45:18.308523 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:18.308572 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":117,"threads_started":1}
I20260809 00:45:19.106855 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:19.106486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:55779)
0809 00:45:19.106678 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:19.106802 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":101,"threads_started":1}
I20260809 00:45:19.175590 20866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:19.175229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:33415)
0809 00:45:19.175377 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:19.175529 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260809 00:45:19.309438 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:19.309207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:51209)
0809 00:45:19.309344 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:19.309389 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260809 00:45:20.107920 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:20.107598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:44057)
0809 00:45:20.107822 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:20.107869 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":84,"threads_started":1}
I20260809 00:45:20.176438 20870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:20.176136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:36139)
0809 00:45:20.176331 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:20.176386 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":97,"threads_started":1}
I20260809 00:45:20.310294 20871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:20.310043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:35079)
0809 00:45:20.310193 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:20.310237 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260809 00:45:21.108933 20872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:21.108680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.193:43765)
0809 00:45:21.108834 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:21.108884 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
I20260809 00:45:21.177420 20874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:21.177105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.194:53889)
0809 00:45:21.177296 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:21.177353 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":93,"threads_started":1}
I20260809 00:45:21.306360 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 20145
I20260809 00:45:21.313565 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:21.311719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39737 (local address 127.30.104.195:48765)
0809 00:45:21.313436 (+ 1717us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:21.313497 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:39737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1690,"thread_start_us":64,"threads_started":1}
I20260809 00:45:21.321785 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 20388
I20260809 00:45:21.333596 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 20585
2026-08-09T00:45:21Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18625 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:45:21 dist-test-slave-c12x krb5kdc[20885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:45:21 dist-test-slave-c12x krb5kdc[20885](info): set up 2 sockets
Aug 09 00:45:21 dist-test-slave-c12x krb5kdc[20885](info): commencing operation
krb5kdc: starting...
W20260809 00:45:22.575610 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.215s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:45:22 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:45:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:45:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:45:22.708861 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37297
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:22.798864 20903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:22.799083 20903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:22.799106 20903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:22.800264 20903 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:45:22.800307 20903 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:45:22.800319 20903 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:45:22.800330 20903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:45:22.800343 20903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:22.800357 20903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:22.800369 20903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:45:22.800381 20903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:45:22.801846 20903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:22.802057 20903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:22.802228 20903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:22.807472 20912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:22.807479 20915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:22.807552 20913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:22.807502 20903 server_base.cc:1034] running on GCE node
I20260809 00:45:22.807922 20903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:22.808303 20903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:22.809501 20903 hybrid_clock.cc:648] HybridClock initialized: now 1786236322809466 us; error 58 us; skew 500 ppm
Aug 09 00:45:22 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236322, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:22.812224 20903 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:45:22.812754 20903 webserver.cc:460] Webserver started at http://127.30.104.254:37809/ using document root <none> and password file <none>
I20260809 00:45:22.813063 20903 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:22.813143 20903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:22.813302 20903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:22.814427 20903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "a476f00314fe4d93b40f60678a9c90cd"
format_stamp: "Formatted at 2026-08-09 00:45:22 on dist-test-slave-c12x"
server_key: "385d150bcd2a75e716d03bc513fc969f"
server_key_iv: "b5082e02688080edf3a219613d97abeb"
server_key_version: "encryptionkey@0"
I20260809 00:45:22.814819 20903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "a476f00314fe4d93b40f60678a9c90cd"
format_stamp: "Formatted at 2026-08-09 00:45:22 on dist-test-slave-c12x"
server_key: "385d150bcd2a75e716d03bc513fc969f"
server_key_iv: "b5082e02688080edf3a219613d97abeb"
server_key_version: "encryptionkey@0"
I20260809 00:45:22.816411 20903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:45:22.817272 20923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:22.817453 20903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:22.817595 20903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a476f00314fe4d93b40f60678a9c90cd"
format_stamp: "Formatted at 2026-08-09 00:45:22 on dist-test-slave-c12x"
server_key: "385d150bcd2a75e716d03bc513fc969f"
server_key_iv: "b5082e02688080edf3a219613d97abeb"
server_key_version: "encryptionkey@0"
I20260809 00:45:22.817679 20903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:22.838351 20903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:22.840478 20903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:22.840662 20903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:22.844722 20903 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37297
I20260809 00:45:22.844766 20999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37297 every 8 connection(s)
I20260809 00:45:22.845175 20903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:45:22.845636 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 20903
I20260809 00:45:22.845757 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:45:22.845944 31139 external_mini_cluster.cc:1428] Setting key 12773f21e7005fcd3cfa11ef39d6bcb5
I20260809 00:45:22.846378 21000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:22.849949 21000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Bootstrap starting.
I20260809 00:45:22.850617 21000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:22.850966 21000 log.cc:824] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:22.851756 21000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: No bootstrap required, opened a new log
Aug 09 00:45:22 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:22.854295 21000 raft_consensus.cc:348] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:45:22.854488 21000 raft_consensus.cc:374] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:22.854542 21000 raft_consensus.cc:729] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a476f00314fe4d93b40f60678a9c90cd, State: Initialized, Role: FOLLOWER
I20260809 00:45:22.854722 21000 consensus_queue.cc:260] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:45:22.854827 21000 raft_consensus.cc:388] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:45:22.854878 21000 raft_consensus.cc:482] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:45:22.854955 21000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:22.855592 21000 raft_consensus.cc:504] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:45:22.855717 21000 leader_election.cc:302] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a476f00314fe4d93b40f60678a9c90cd; no voters:
I20260809 00:45:22.855919 21000 leader_election.cc:288] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:45:22.855981 21009 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:45:22.856161 21009 raft_consensus.cc:686] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 1 LEADER]: Becoming Leader. State: Replica: a476f00314fe4d93b40f60678a9c90cd, State: Running, Role: LEADER
I20260809 00:45:22.856215 21000 sys_catalog.cc:564] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:45:22.856411 21009 consensus_queue.cc:237] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:45:22.856858 21010 sys_catalog.cc:455] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a476f00314fe4d93b40f60678a9c90cd. Latest consensus state: current_term: 1 leader_uuid: "a476f00314fe4d93b40f60678a9c90cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } } }
I20260809 00:45:22.856990 21010 sys_catalog.cc:458] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:22.856853 21012 sys_catalog.cc:455] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a476f00314fe4d93b40f60678a9c90cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } } }
I20260809 00:45:22.857537 21012 sys_catalog.cc:458] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:22.857749 21019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:45:22.858373 21019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:45:22.858608 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:22.846573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56334 (local address 127.30.104.254:37297)
0809 00:45:22.846788 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:22.846797 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:22.846831 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:45:22.847671 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:22.847681 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:22.847748 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:22.847905 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:22.848528 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:22.849903 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:22.850734 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:22.851337 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:22.851487 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:22.854150 (+ 2663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:22.854155 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:22.854155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:22.854167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:22.856362 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:22.856783 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:22.856786 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:22.856786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:22.856899 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:22.857139 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:22.857140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:22.857141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:22.857507 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:22.857688 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:22.858221 (+ 533us) server_negotiation.cc:299] Negotiation successful
0809 00:45:22.858332 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":102,"threads_started":1}
I20260809 00:45:22.859972 21019 catalog_manager.cc:1347] Generated new cluster ID: 061f3052e5d346a9bf5cd014d47cdd63
I20260809 00:45:22.860037 21019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:45:22.888450 21019 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:45:22.889000 21019 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:45:22.893087 21019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Generated new TSK 0
I20260809 00:45:22.893282 21019 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:45:22.936817 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37297
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:23.041471 21039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:23.041710 21039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:23.041735 21039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:23.042917 21039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:23.042966 21039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:23.042994 21039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:45:23.044565 21039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:23.044839 21039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:23.045102 21039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:23.050488 21048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:23.050709 21053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:23.050709 21051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:23.050760 21039 server_base.cc:1034] running on GCE node
I20260809 00:45:23.051044 21039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:23.051308 21039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:23.052486 21039 hybrid_clock.cc:648] HybridClock initialized: now 1786236323052473 us; error 33 us; skew 500 ppm
Aug 09 00:45:23 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:23.055122 21039 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:45:23.055699 21039 webserver.cc:460] Webserver started at http://127.30.104.193:40299/ using document root <none> and password file <none>
I20260809 00:45:23.055974 21039 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:23.056047 21039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:23.056188 21039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:23.057436 21039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "3806f5b0340e4d84888fe51a6c72ef70"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "4e26b744f82f11402d98437ea651a843"
server_key_iv: "87cfffeefecbb8b49dbf087655645e34"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.057883 21039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "3806f5b0340e4d84888fe51a6c72ef70"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "4e26b744f82f11402d98437ea651a843"
server_key_iv: "87cfffeefecbb8b49dbf087655645e34"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.059288 21039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:45:23.060171 21062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.060354 21039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:23.060458 21039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "3806f5b0340e4d84888fe51a6c72ef70"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "4e26b744f82f11402d98437ea651a843"
server_key_iv: "87cfffeefecbb8b49dbf087655645e34"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.060549 21039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:23.071193 21039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:23.073701 21039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:23.073983 21039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:23.074278 21039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:23.074643 21039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:23.074703 21039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.074748 21039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:23.074776 21039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.081096 21039 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:45405
I20260809 00:45:23.081180 21230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:45405 every 8 connection(s)
I20260809 00:45:23.081594 21039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:45:23.084583 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 21039
I20260809 00:45:23.084689 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:45:23.084926 31139 external_mini_cluster.cc:1428] Setting key 640c9d6ed2053b6a07b269548c7b8269
Aug 09 00:45:23 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:23.092880 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.082469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:38351 (local address 127.30.104.254:37297)
0809 00:45:23.082523 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.082525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.082751 (+ 226us) server_negotiation.cc:407] Connection header received
0809 00:45:23.083307 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.083308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.083322 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.083375 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:23.084415 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.085097 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.086250 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.086478 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.086573 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.089250 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:23.089252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:23.089252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:23.089261 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:23.091020 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:23.091601 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:23.091603 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:23.091603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:23.091680 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:23.092039 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:23.092040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:23.092041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:23.092220 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:23.092305 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.092650 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.092763 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:45:23.092967 21233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.082384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:38351)
0809 00:45:23.082562 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.082645 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:23.083191 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:23.083486 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:23.083490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:23.084098 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:23.084299 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.084304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.085192 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.085193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.086154 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.086157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.086587 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.086588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.086726 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.087353 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:23.087364 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:23.089139 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:23.091128 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:23.091136 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:23.091138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:23.091509 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:23.091786 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:23.091787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:23.091787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:23.091916 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:23.092320 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:23.092328 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:23.092477 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 00:45:23.092577 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 00:45:23.092730 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260809 00:45:23.093607 21231 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37297
I20260809 00:45:23.093716 21231 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:23.093928 21231 heartbeater.cc:511] Master 127.30.104.254:37297 requested a full tablet report, sending...
I20260809 00:45:23.094596 20950 ts_manager.cc:194] Registered new tserver with Master: 3806f5b0340e4d84888fe51a6c72ef70 (127.30.104.193:45405)
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
I20260809 00:45:23.095597 20950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:38351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:45:23.116678 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37297
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:23.202642 21237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:23.202872 21237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:23.202898 21237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:23.204221 21237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:23.204269 21237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:23.204298 21237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:45:23.205848 21237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:23.206064 21237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:23.206262 21237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:23.211638 21246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:23.211637 21243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:23.211762 21242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:23.212024 21237 server_base.cc:1034] running on GCE node
I20260809 00:45:23.212265 21237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:23.212565 21237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:23.213738 21237 hybrid_clock.cc:648] HybridClock initialized: now 1786236323213722 us; error 33 us; skew 500 ppm
Aug 09 00:45:23 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:23.216522 21237 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:45:23.217000 21237 webserver.cc:460] Webserver started at http://127.30.104.194:42107/ using document root <none> and password file <none>
I20260809 00:45:23.217278 21237 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:23.217356 21237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:23.217514 21237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:23.218642 21237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "669d9667d5d140eb8fef9cd5a07708f0"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "029aeafbe533300056d11d63461e8ec8"
server_key_iv: "eaf429e65c10608ee33bbfd7f99b7da5"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.219033 21237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "669d9667d5d140eb8fef9cd5a07708f0"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "029aeafbe533300056d11d63461e8ec8"
server_key_iv: "eaf429e65c10608ee33bbfd7f99b7da5"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.220785 21237 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 00:45:23.221840 21255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.222097 21237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:23.222213 21237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "669d9667d5d140eb8fef9cd5a07708f0"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "029aeafbe533300056d11d63461e8ec8"
server_key_iv: "eaf429e65c10608ee33bbfd7f99b7da5"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.222294 21237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:23.231664 21237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:23.234247 21237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:23.234431 21237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:23.234848 21237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:23.235554 21237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:23.235623 21237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.235671 21237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:23.235718 21237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.242578 21237 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:44669
I20260809 00:45:23.242626 21399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:44669 every 8 connection(s)
I20260809 00:45:23.243335 21237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:45:23.243649 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 21237
I20260809 00:45:23.243779 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:45:23.244035 31139 external_mini_cluster.cc:1428] Setting key 28b0c0d1cf191a2a7cfb37496c34a4e2
Aug 09 00:45:23 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
I20260809 00:45:23.254180 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.244492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:46965 (local address 127.30.104.254:37297)
0809 00:45:23.244567 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.244569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.244917 (+ 348us) server_negotiation.cc:407] Connection header received
0809 00:45:23.245382 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.245384 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.245396 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.245461 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:23.246465 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.247146 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.248243 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.248464 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.248544 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.250920 (+ 2376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:23.250922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:23.250922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:23.250930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:23.252536 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:23.253003 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:23.253005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:23.253005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:23.253070 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:23.253341 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:23.253342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:23.253343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:23.253507 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:23.253676 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.253979 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.254080 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:45:23.254385 21402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.244467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:46965)
0809 00:45:23.244664 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.244704 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:23.245274 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:23.245541 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:23.245546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:23.246150 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:23.246358 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.246363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.247258 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.247259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.248145 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.248148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.248552 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.248553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.248686 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.249327 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:23.249335 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:23.250814 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:23.252666 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:23.252673 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:23.252676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:23.252919 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:23.253154 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:23.253154 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:23.253155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:23.253271 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:23.253663 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:23.253671 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:23.253833 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 00:45:23.253955 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:45:23.254143 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":100,"threads_started":1}
I20260809 00:45:23.255007 21400 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37297
I20260809 00:45:23.255136 21400 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:23.255355 21400 heartbeater.cc:511] Master 127.30.104.254:37297 requested a full tablet report, sending...
I20260809 00:45:23.255960 20950 ts_manager.cc:194] Registered new tserver with Master: 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669)
I20260809 00:45:23.256412 20950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:46965
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:45:23.275682 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37297
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:23.364979 21405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:23.365211 21405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:23.365237 21405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:23.366468 21405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:23.366518 21405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:23.366547 21405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:45:23.368181 21405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:23.368456 21405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:23.368734 21405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:23.374341 21414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:23.374339 21418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:23.374377 21405 server_base.cc:1034] running on GCE node
W20260809 00:45:23.374312 21415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:23.374801 21405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:23.375093 21405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:23.376267 21405 hybrid_clock.cc:648] HybridClock initialized: now 1786236323376260 us; error 37 us; skew 500 ppm
Aug 09 00:45:23 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:23.378919 21405 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:45:23.379408 21405 webserver.cc:460] Webserver started at http://127.30.104.195:41331/ using document root <none> and password file <none>
I20260809 00:45:23.379716 21405 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:23.379791 21405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:23.379949 21405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:45:23.381067 21405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "1a8c339eb0e3eb48af685ee29b2d2619"
server_key_iv: "a2bb24bba2475d79b733f878da8fb20f"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.381474 21405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "1a8c339eb0e3eb48af685ee29b2d2619"
server_key_iv: "a2bb24bba2475d79b733f878da8fb20f"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.382889 21405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:45:23.383795 21428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.384019 21405 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:23.384143 21405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "1a8c339eb0e3eb48af685ee29b2d2619"
server_key_iv: "a2bb24bba2475d79b733f878da8fb20f"
server_key_version: "encryptionkey@0"
I20260809 00:45:23.384227 21405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:23.401918 21405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:23.404474 21405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:23.404654 21405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:23.404909 21405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:23.405257 21405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:23.405314 21405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.405359 21405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:23.405385 21405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:23.411670 21405 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:42095
I20260809 00:45:23.411731 21588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:42095 every 8 connection(s)
I20260809 00:45:23.412091 21405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:45:23.412462 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 21405
I20260809 00:45:23.412562 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:45:23.412792 31139 external_mini_cluster.cc:1428] Setting key 30a619b49ac9c162854274c8b1070c33
Aug 09 00:45:23 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236323, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:23.421977 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.412930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:36797 (local address 127.30.104.254:37297)
0809 00:45:23.412989 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.412992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.413184 (+ 192us) server_negotiation.cc:407] Connection header received
0809 00:45:23.413776 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.413777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.413790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.413844 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:23.414854 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.415382 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.416451 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.416675 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.416761 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.419071 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:23.419072 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:23.419073 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:23.419080 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:23.420613 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:23.421069 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:23.421070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:23.421070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:23.421125 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:23.421362 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:23.421363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:23.421363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:23.421471 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:23.421524 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.421826 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.421906 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 00:45:23.422102 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.412908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:36797)
0809 00:45:23.413062 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.413107 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:23.413660 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:23.413917 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:23.413921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:23.414541 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:23.414748 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.414753 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.415531 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.415532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.416347 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.416349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.416770 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.416771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.416911 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.417493 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:23.417502 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:23.418962 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:23.420730 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:23.420737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:23.420738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:23.421001 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:23.421190 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:23.421191 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:23.421191 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:23.421296 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:23.421528 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:23.421532 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:23.421681 (+ 149us) client_negotiation.cc:769] Sending connection context
0809 00:45:23.421767 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:45:23.421897 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260809 00:45:23.422654 21590 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37297
I20260809 00:45:23.422751 21590 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:23.422948 21590 heartbeater.cc:511] Master 127.30.104.254:37297 requested a full tablet report, sending...
I20260809 00:45:23.423399 20950 ts_manager.cc:194] Registered new tserver with Master: 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
I20260809 00:45:23.423885 20950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:36797
I20260809 00:45:23.424520 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:45:23.432996 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.426011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.30.104.254:37297)
0809 00:45:23.426099 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.426101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.426185 (+ 84us) server_negotiation.cc:407] Connection header received
0809 00:45:23.426227 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.426228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.426237 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.426278 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:23.426961 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.427524 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.428254 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.428457 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.428524 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.429806 (+ 1282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:23.429807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:23.429807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:23.429814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:23.431290 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:23.431846 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:23.431849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:23.431850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:23.431980 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:23.432269 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:23.432271 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:23.432271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:23.432455 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:23.432546 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.432661 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.432767 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:45:23.435075 20949 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:45:23.436861 20949 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:45:23.443885 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.439821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:42095 (local address 127.0.0.1:41140)
0809 00:45:23.440159 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.440169 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:23.440218 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:23.440513 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:23.440513 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:23.440515 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:23.440563 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.440564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.441456 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.441457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.442669 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.442672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.443546 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.443547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.443627 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.443642 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:45:23.443745 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 00:45:23.443807 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260809 00:45:23.443894 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.439942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.30.104.195:42095)
0809 00:45:23.440130 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.440135 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.440300 (+ 165us) server_negotiation.cc:407] Connection header received
0809 00:45:23.440344 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.440346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.440361 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.440434 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:23.440641 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.441348 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.442774 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.443410 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.443543 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.443608 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.443763 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.443806 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":113,"threads_started":1}
I20260809 00:45:23.444229 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.439614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44669 (local address 127.0.0.1:38560)
0809 00:45:23.439991 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.440003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:23.440119 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:23.440420 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:23.440424 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:23.440427 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:23.440522 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.440524 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.441881 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.441882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.442940 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.442942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.443933 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.443934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.443994 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.443997 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:45:23.444049 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:45:23.444080 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":44,"thread_start_us":99,"threads_started":1}
I20260809 00:45:23.444250 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.439726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38560 (local address 127.30.104.194:44669)
0809 00:45:23.439939 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.439945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.440088 (+ 143us) server_negotiation.cc:407] Connection header received
0809 00:45:23.440214 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.440217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.440235 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.440316 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:23.440628 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.441762 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.443023 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.443766 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.443846 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.443904 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.444127 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.444175 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
I20260809 00:45:23.444398 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.439548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45405 (local address 127.0.0.1:38128)
0809 00:45:23.439686 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.439727 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:23.439799 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:23.440453 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:23.440454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:23.440456 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:23.440533 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.440535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.441909 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.441910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.442996 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.442999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.443915 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.443916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.443993 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.444000 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 00:45:23.444030 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 00:45:23.444072 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":16,"thread_start_us":92,"threads_started":1}
I20260809 00:45:23.444828 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.439966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38128 (local address 127.30.104.193:45405)
0809 00:45:23.440274 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.440279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.440289 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:45:23.440332 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.440334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.440350 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.440646 (+ 296us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:23.440726 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.441787 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.443088 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.443819 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.444589 (+ 770us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.444664 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.444690 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.444742 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":95,"threads_started":1}
I20260809 00:45:23.444972 21323 tablet_service.cc:1467] Processing CreateTablet for tablet 55c03d51b5764abbaf5f3ad8360e1835 (DEFAULT_TABLE table=test-table [id=fd7b5f8e4ed94f14843b924b0e43802b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:23.444921 21501 tablet_service.cc:1467] Processing CreateTablet for tablet 55c03d51b5764abbaf5f3ad8360e1835 (DEFAULT_TABLE table=test-table [id=fd7b5f8e4ed94f14843b924b0e43802b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:23.445320 21501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c03d51b5764abbaf5f3ad8360e1835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:23.445319 21323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c03d51b5764abbaf5f3ad8360e1835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:23.445402 21137 tablet_service.cc:1467] Processing CreateTablet for tablet 55c03d51b5764abbaf5f3ad8360e1835 (DEFAULT_TABLE table=test-table [id=fd7b5f8e4ed94f14843b924b0e43802b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:23.445649 21137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 55c03d51b5764abbaf5f3ad8360e1835. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:23.448311 21614 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Bootstrap starting.
I20260809 00:45:23.448694 21618 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: Bootstrap starting.
I20260809 00:45:23.449039 21614 tablet_bootstrap.cc:654] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:23.449375 21618 tablet_bootstrap.cc:654] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:23.449733 21614 log.cc:824] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:23.449997 21618 log.cc:824] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:23.450407 21615 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Bootstrap starting.
I20260809 00:45:23.450631 21614 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: No bootstrap required, opened a new log
I20260809 00:45:23.450718 21614 ts_tablet_manager.cc:1397] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:45:23.450798 21618 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: No bootstrap required, opened a new log
I20260809 00:45:23.450870 21618 ts_tablet_manager.cc:1397] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:45:23.452102 21615 tablet_bootstrap.cc:654] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:23.452402 21614 raft_consensus.cc:348] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.452579 21614 raft_consensus.cc:374] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:23.452625 21614 raft_consensus.cc:729] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3806f5b0340e4d84888fe51a6c72ef70, State: Initialized, Role: FOLLOWER
I20260809 00:45:23.452677 21615 log.cc:824] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:23.452762 21614 consensus_queue.cc:260] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.453099 21231 heartbeater.cc:503] Master 127.30.104.254:37297 was elected leader, sending a full tablet report...
I20260809 00:45:23.453096 21618 raft_consensus.cc:348] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.453274 21618 raft_consensus.cc:374] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:23.453320 21618 raft_consensus.cc:729] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669d9667d5d140eb8fef9cd5a07708f0, State: Initialized, Role: FOLLOWER
I20260809 00:45:23.453438 21614 ts_tablet_manager.cc:1428] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:45:23.453461 21615 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: No bootstrap required, opened a new log
I20260809 00:45:23.453521 21615 ts_tablet_manager.cc:1397] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:45:23.453547 21618 consensus_queue.cc:260] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.453838 21618 ts_tablet_manager.cc:1428] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:45:23.453886 21400 heartbeater.cc:503] Master 127.30.104.254:37297 was elected leader, sending a full tablet report...
I20260809 00:45:23.455658 21615 raft_consensus.cc:348] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.455854 21615 raft_consensus.cc:374] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:23.455920 21615 raft_consensus.cc:729] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d8fb51a1eca4e6883fe99a18e04e7ee, State: Initialized, Role: FOLLOWER
I20260809 00:45:23.456048 21615 consensus_queue.cc:260] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.456331 21615 ts_tablet_manager.cc:1428] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:45:23.456425 21590 heartbeater.cc:503] Master 127.30.104.254:37297 was elected leader, sending a full tablet report...
I20260809 00:45:23.460775 21622 raft_consensus.cc:482] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:23.460907 21622 raft_consensus.cc:504] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.461292 21622 leader_election.cc:288] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
I20260809 00:45:23.464241 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.461536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:57749 (local address 127.30.104.194:44669)
0809 00:45:23.461580 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.461582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.461593 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:45:23.461635 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.461636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.461644 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.461700 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:23.461873 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.462355 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.463459 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.463863 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.463979 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.463987 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.464159 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.464189 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:45:23.464268 21233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.461460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44669 (local address 127.30.104.193:57749)
0809 00:45:23.461517 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.461539 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:23.461581 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:23.461722 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:23.461723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:23.461725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:23.461804 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.461806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.462447 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.462448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.463345 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.463348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.463984 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.463986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.464075 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.464079 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:45:23.464145 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:45:23.464188 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:45:23.464603 21349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c03d51b5764abbaf5f3ad8360e1835" candidate_uuid: "3806f5b0340e4d84888fe51a6c72ef70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669d9667d5d140eb8fef9cd5a07708f0" is_pre_election: true
I20260809 00:45:23.464793 21349 raft_consensus.cc:2454] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3806f5b0340e4d84888fe51a6c72ef70 in term 0.
I20260809 00:45:23.464977 21626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.461579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:42095 (local address 127.30.104.193:39621)
0809 00:45:23.461791 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.461804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:23.461857 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:23.461982 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:23.461983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:23.461985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:23.462062 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.462064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.462767 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.462768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.463765 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:23.463767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.464702 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.464703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:23.464780 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.464785 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:45:23.464850 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:45:23.464890 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":136,"threads_started":1}
I20260809 00:45:23.465003 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.461828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:39621 (local address 127.30.104.195:42095)
0809 00:45:23.461879 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.461881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.461890 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:45:23.461915 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.461916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.461925 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.461980 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:23.462156 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.462667 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.463862 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.464579 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.464654 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.464663 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.464884 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.464930 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:45:23.465051 21066 leader_election.cc:302] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3806f5b0340e4d84888fe51a6c72ef70, 669d9667d5d140eb8fef9cd5a07708f0; no voters:
I20260809 00:45:23.465255 21622 raft_consensus.cc:2781] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:45:23.465317 21622 raft_consensus.cc:482] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:45:23.465369 21622 raft_consensus.cc:3037] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:23.465389 21527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c03d51b5764abbaf5f3ad8360e1835" candidate_uuid: "3806f5b0340e4d84888fe51a6c72ef70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" is_pre_election: true
I20260809 00:45:23.465561 21527 raft_consensus.cc:2454] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3806f5b0340e4d84888fe51a6c72ef70 in term 0.
I20260809 00:45:23.466279 21622 raft_consensus.cc:504] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.466426 21622 leader_election.cc:288] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 1 election: Requested vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
I20260809 00:45:23.466599 21527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c03d51b5764abbaf5f3ad8360e1835" candidate_uuid: "3806f5b0340e4d84888fe51a6c72ef70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee"
I20260809 00:45:23.466596 21349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "55c03d51b5764abbaf5f3ad8360e1835" candidate_uuid: "3806f5b0340e4d84888fe51a6c72ef70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669d9667d5d140eb8fef9cd5a07708f0"
I20260809 00:45:23.466677 21349 raft_consensus.cc:3037] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:23.466677 21527 raft_consensus.cc:3037] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:23.467682 21527 raft_consensus.cc:2454] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3806f5b0340e4d84888fe51a6c72ef70 in term 1.
I20260809 00:45:23.467682 21349 raft_consensus.cc:2454] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3806f5b0340e4d84888fe51a6c72ef70 in term 1.
I20260809 00:45:23.467855 21066 leader_election.cc:302] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d8fb51a1eca4e6883fe99a18e04e7ee, 3806f5b0340e4d84888fe51a6c72ef70; no voters:
I20260809 00:45:23.467979 21622 raft_consensus.cc:2781] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:45:23.468153 21622 raft_consensus.cc:686] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 LEADER]: Becoming Leader. State: Replica: 3806f5b0340e4d84888fe51a6c72ef70, State: Running, Role: LEADER
I20260809 00:45:23.468268 21622 consensus_queue.cc:237] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:23.469206 20950 catalog_manager.cc:5495] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3806f5b0340e4d84888fe51a6c72ef70 (127.30.104.193). New cstate: current_term: 1 leader_uuid: "3806f5b0340e4d84888fe51a6c72ef70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } health_report { overall_health: UNKNOWN } } }
I20260809 00:45:23.478960 21610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.476816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.30.104.193:45405)
0809 00:45:23.476883 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:23.476885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:23.476892 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:45:23.476924 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:23.476925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:23.476941 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:23.477010 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:45:23.477123 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.477593 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.478297 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:23.478485 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:23.478542 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:23.478678 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:45:23.478776 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:45:23.478838 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:23.478869 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 00:45:23.478885 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:45:23.480907 21349 raft_consensus.cc:1261] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 1 FOLLOWER]: Refusing update from remote peer 3806f5b0340e4d84888fe51a6c72ef70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:45:23.480948 21527 raft_consensus.cc:1261] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 FOLLOWER]: Refusing update from remote peer 3806f5b0340e4d84888fe51a6c72ef70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:45:23.481312 21627 consensus_queue.cc:1035] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:45:23.481459 21622 consensus_queue.cc:1035] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:45:23.484099 21634 mvcc.cc:205] Tried to move back new op lower bound from 7316423980974825472 to 7316423980926877696. Current Snapshot: MvccSnapshot[applied={T|T < 7316423980974825472}]
I20260809 00:45:23.484697 21636 mvcc.cc:205] Tried to move back new op lower bound from 7316423980974825472 to 7316423980926877696. Current Snapshot: MvccSnapshot[applied={T|T < 7316423980974825472}]
I20260809 00:45:23.486199 20949 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:45:23.486320 20949 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:45:23.487084 20949 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Sending DeleteTablet for 3 replicas of tablet 55c03d51b5764abbaf5f3ad8360e1835
I20260809 00:45:23.487397 21137 tablet_service.cc:1514] Processing DeleteTablet for tablet 55c03d51b5764abbaf5f3ad8360e1835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:23 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38128
I20260809 00:45:23.487527 21323 tablet_service.cc:1514] Processing DeleteTablet for tablet 55c03d51b5764abbaf5f3ad8360e1835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:23 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38560
I20260809 00:45:23.487569 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 21039
I20260809 00:45:23.495587 21501 tablet_service.cc:1514] Processing DeleteTablet for tablet 55c03d51b5764abbaf5f3ad8360e1835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:23 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:41140
I20260809 00:45:23.496421 21647 tablet_replica.cc:330] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: stopping tablet replica
I20260809 00:45:23.496562 21647 raft_consensus.cc:2229] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:45:23.496687 21647 raft_consensus.cc:2258] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:23.496675 21649 tablet_replica.cc:330] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: stopping tablet replica
I20260809 00:45:23.497090 21649 raft_consensus.cc:2229] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:45:23.497308 21649 raft_consensus.cc:2258] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:23.498211 21647 ts_tablet_manager.cc:1905] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:23.499604 21649 ts_tablet_manager.cc:1905] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:23.500560 21647 ts_tablet_manager.cc:1918] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:23.500643 21647 log.cc:1196] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/55c03d51b5764abbaf5f3ad8360e1835
I20260809 00:45:23.500893 21647 ts_tablet_manager.cc:1939] T 55c03d51b5764abbaf5f3ad8360e1835 P 669d9667d5d140eb8fef9cd5a07708f0: Deleting consensus metadata
I20260809 00:45:23.501441 20930 catalog_manager.cc:4841] TS 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669): tablet 55c03d51b5764abbaf5f3ad8360e1835 (table test-table [id=fd7b5f8e4ed94f14843b924b0e43802b]) successfully deleted
I20260809 00:45:23.502025 21649 ts_tablet_manager.cc:1918] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:23.502113 21649 log.cc:1196] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/55c03d51b5764abbaf5f3ad8360e1835
I20260809 00:45:23.502384 21649 ts_tablet_manager.cc:1939] T 55c03d51b5764abbaf5f3ad8360e1835 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Deleting consensus metadata
I20260809 00:45:23.502841 20930 catalog_manager.cc:4841] TS 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095): tablet 55c03d51b5764abbaf5f3ad8360e1835 (table test-table [id=fd7b5f8e4ed94f14843b924b0e43802b]) successfully deleted
W20260809 00:45:23.506170 20926 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:45405 (error 108)
I20260809 00:45:23.506371 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 21237
I20260809 00:45:23.510725 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:23.506599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45405 (local address 127.0.0.1:38146)
0809 00:45:23.510601 (+ 4002us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:23.510674 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3968}
W20260809 00:45:23.517642 20926 catalog_manager.cc:4584] TS 3806f5b0340e4d84888fe51a6c72ef70 (127.30.104.193:45405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 55c03d51b5764abbaf5f3ad8360e1835: Network error: Client connection negotiation failed: client connection to 127.30.104.193:45405: connect: Connection refused (error 111)
I20260809 00:45:23.521288 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 21405
I20260809 00:45:23.541513 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 20903
I20260809 00:45:43.555655 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37297
--webserver_interface=127.30.104.254
--webserver_port=37809
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:45:43.644660 21651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:43.644927 21651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:43.644964 21651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:43.646133 21651 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:45:43.646206 21651 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:45:43.646229 21651 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:45:43.646248 21651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:45:43.646279 21651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:43.646309 21651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:43.646342 21651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:45:43.646370 21651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:45:43.648001 21651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=37809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:43.648327 21651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:43.648581 21651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:43.658867 21661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:43.658936 21664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:43.658903 21662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:43.658916 21651 server_base.cc:1034] running on GCE node
I20260809 00:45:43.659332 21651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:43.659708 21651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:43.660876 21651 hybrid_clock.cc:648] HybridClock initialized: now 1786236343660863 us; error 31 us; skew 500 ppm
Aug 09 00:45:43 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236343, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:43.663630 21651 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:45:43.664132 21651 webserver.cc:460] Webserver started at http://127.30.104.254:37809/ using document root <none> and password file <none>
I20260809 00:45:43.664357 21651 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:43.664412 21651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:43.665827 21651 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:43.666508 21675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:43.666738 21651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:43.666867 21651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a476f00314fe4d93b40f60678a9c90cd"
format_stamp: "Formatted at 2026-08-09 00:45:22 on dist-test-slave-c12x"
server_key: "385d150bcd2a75e716d03bc513fc969f"
server_key_iv: "b5082e02688080edf3a219613d97abeb"
server_key_version: "encryptionkey@0"
I20260809 00:45:43.667326 21651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:43.690024 21651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:43.692505 21651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:43.692682 21651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:43.697263 21651 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37297
I20260809 00:45:43.697299 21752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37297 every 8 connection(s)
I20260809 00:45:43.697743 21651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:45:43.699550 21753 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:45:43.700760 21753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Bootstrap starting.
I20260809 00:45:43.702586 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 21651
I20260809 00:45:43.702950 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:45405
--local_ip_for_outbound_sockets=127.30.104.193
--tserver_master_addrs=127.30.104.254:37297
--webserver_port=40299
--webserver_interface=127.30.104.193
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:45:43.709739 21753 log.cc:824] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:43.712018 21753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:45:43.712378 21753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Bootstrap complete.
I20260809 00:45:43.714808 21753 raft_consensus.cc:348] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:45:43.715093 21753 raft_consensus.cc:729] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a476f00314fe4d93b40f60678a9c90cd, State: Initialized, Role: FOLLOWER
I20260809 00:45:43.715315 21753 consensus_queue.cc:260] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:45:43.715426 21753 raft_consensus.cc:388] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:45:43.715514 21753 raft_consensus.cc:482] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:45:43.715564 21753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 1 FOLLOWER]: Advancing to term 2
I20260809 00:45:43.716662 21753 raft_consensus.cc:504] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:45:43.716796 21753 leader_election.cc:302] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a476f00314fe4d93b40f60678a9c90cd; no voters:
I20260809 00:45:43.717016 21753 leader_election.cc:288] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 00:45:43.717056 21758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 2 FOLLOWER]: Leader election won for term 2
I20260809 00:45:43.717185 21758 raft_consensus.cc:686] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 2 LEADER]: Becoming Leader. State: Replica: a476f00314fe4d93b40f60678a9c90cd, State: Running, Role: LEADER
I20260809 00:45:43.717324 21753 sys_catalog.cc:564] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:45:43.717344 21758 consensus_queue.cc:237] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:45:43.717759 21759 sys_catalog.cc:455] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a476f00314fe4d93b40f60678a9c90cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } } }
I20260809 00:45:43.717849 21759 sys_catalog.cc:458] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:43.718118 21760 sys_catalog.cc:455] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a476f00314fe4d93b40f60678a9c90cd. Latest consensus state: current_term: 2 leader_uuid: "a476f00314fe4d93b40f60678a9c90cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } } }
I20260809 00:45:43.718204 21760 sys_catalog.cc:458] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: This master's current role is: LEADER
I20260809 00:45:43.719390 21778 catalog_manager.cc:1259] Loaded cluster ID: 061f3052e5d346a9bf5cd014d47cdd63
I20260809 00:45:43.719473 21778 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: loading cluster ID for follower catalog manager: success
I20260809 00:45:43.720710 21778 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: acquiring CA information for follower catalog manager: success
I20260809 00:45:43.721158 21778 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:45:43.721243 21766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:45:43.721629 21766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:45:43.721694 21766 catalog_manager.cc:1259] Loaded cluster ID: 061f3052e5d346a9bf5cd014d47cdd63
I20260809 00:45:43.721747 21766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:45:43.721907 21766 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:45:43.721982 21766 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Loaded TSK: 0
I20260809 00:45:43.739063 21766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Generated new TSK 1
I20260809 00:45:43.739193 21766 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:45:43.792311 21756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:43.792613 21756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:43.792665 21756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:43.794008 21756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:43.794090 21756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:43.794138 21756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:45:43.795883 21756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:45405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=40299
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:43.796180 21756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:43.796443 21756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:43.802076 21792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:43.802093 21788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:43.802358 21756 server_base.cc:1034] running on GCE node
W20260809 00:45:43.802104 21789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:43.802707 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:43.803012 21756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:43.804183 21756 hybrid_clock.cc:648] HybridClock initialized: now 1786236343804171 us; error 27 us; skew 500 ppm
Aug 09 00:45:43 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236343, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:43.806974 21756 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:45:43.807582 21756 webserver.cc:460] Webserver started at http://127.30.104.193:40299/ using document root <none> and password file <none>
I20260809 00:45:43.807866 21756 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:43.807931 21756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:43.809849 21756 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:45:43.810740 21803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:43.810933 21756 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:43.811048 21756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "3806f5b0340e4d84888fe51a6c72ef70"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "4e26b744f82f11402d98437ea651a843"
server_key_iv: "87cfffeefecbb8b49dbf087655645e34"
server_key_version: "encryptionkey@0"
I20260809 00:45:43.811430 21756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:43.821065 21756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:43.823385 21756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:43.823632 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:43.823906 21756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:43.824484 21819 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:45:43.826221 21756 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:45:43.826275 21756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:45:43.826313 21756 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:45:43.827031 21756 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:45:43.827080 21756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:45:43.827124 21819 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Bootstrap starting.
I20260809 00:45:43.835322 21756 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:45405
I20260809 00:45:43.835392 21978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:45405 every 8 connection(s)
I20260809 00:45:43.835896 21756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:45:43.838836 21819 log.cc:824] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:43.839408 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 21756
I20260809 00:45:43.839773 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:44669
--local_ip_for_outbound_sockets=127.30.104.194
--tserver_master_addrs=127.30.104.254:37297
--webserver_port=42107
--webserver_interface=127.30.104.194
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:45:43.844734 21819 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:45:43.845112 21819 tablet_bootstrap.cc:492] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Bootstrap complete.
Aug 09 00:45:43 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236343, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:43.846406 21819 ts_tablet_manager.cc:1397] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.010s
I20260809 00:45:43.847843 21819 raft_consensus.cc:348] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:43.848213 21819 raft_consensus.cc:729] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3806f5b0340e4d84888fe51a6c72ef70, State: Initialized, Role: FOLLOWER
I20260809 00:45:43.850106 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:43.836954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:36189 (local address 127.30.104.254:37297)
0809 00:45:43.837227 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:43.837237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:43.837331 (+ 94us) server_negotiation.cc:407] Connection header received
0809 00:45:43.838102 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:43.838108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:43.838156 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:43.838312 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:43.839101 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.840764 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.842358 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.843074 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.843240 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:43.845941 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:43.845946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:43.845946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:43.845960 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:43.847949 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:43.848563 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:43.848574 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:43.848574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:43.848678 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:43.848988 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:43.848989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:43.848990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:43.849269 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:43.849378 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:43.849717 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 00:45:43.849863 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":85,"threads_started":1}
I20260809 00:45:43.850073 21981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:43.836858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:36189)
0809 00:45:43.837166 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:43.837222 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:43.837820 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:43.838344 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:43.838348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:43.838691 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:43.838893 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:43.838899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.840942 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.840944 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:43.842238 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:43.842243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.843218 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.843219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:43.843378 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:43.844127 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:43.844137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:43.845809 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:43.848091 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:43.848098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:43.848101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:43.848456 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:43.848759 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:43.848761 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:43.848761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:43.848889 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:43.849356 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:43.849365 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:43.849521 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 00:45:43.849637 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 00:45:43.849846 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":131,"threads_started":1}
I20260809 00:45:43.851718 21819 consensus_queue.cc:260] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } } peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } }
I20260809 00:45:43.852252 21819 ts_tablet_manager.cc:1428] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 00:45:43.852734 21979 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37297
I20260809 00:45:43.852845 21979 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:43.853055 21979 heartbeater.cc:511] Master 127.30.104.254:37297 requested a full tablet report, sending...
I20260809 00:45:43.853945 21697 ts_manager.cc:194] Registered new tserver with Master: 3806f5b0340e4d84888fe51a6c72ef70 (127.30.104.193:45405)
I20260809 00:45:43.855221 21697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:36189
I20260809 00:45:43.856196 21979 heartbeater.cc:503] Master 127.30.104.254:37297 was elected leader, sending a full tablet report...
I20260809 00:45:43.863227 21991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:43.854934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43474 (local address 127.30.104.193:45405)
0809 00:45:43.859625 (+ 4691us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:43.859631 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:43.859900 (+ 269us) server_negotiation.cc:407] Connection header received
0809 00:45:43.859945 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:43.859948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:43.859965 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:43.860036 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:43.860329 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.861188 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.862241 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.862731 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.862828 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:43.862893 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:43.863095 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 00:45:43.863139 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4644,"thread_start_us":69,"threads_started":1}
I20260809 00:45:43.863235 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:43.854641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45405 (local address 127.0.0.1:43474)
0809 00:45:43.859624 (+ 4983us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:43.859688 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:43.859797 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:43.860117 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:43.860120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:43.860122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:43.860246 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:43.860249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.861276 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.861278 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:43.862134 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:43.862136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.862828 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.862830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:43.862964 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:43.862975 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 00:45:43.863073 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 00:45:43.863139 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4935,"thread_start_us":125,"threads_started":1}
I20260809 00:45:43.863963 21881 tablet_service.cc:1514] Processing DeleteTablet for tablet 55c03d51b5764abbaf5f3ad8360e1835 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:23 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:43474
I20260809 00:45:43.864548 21992 tablet_replica.cc:330] stopping tablet replica
I20260809 00:45:43.864687 21992 raft_consensus.cc:2229] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:45:43.864766 21992 raft_consensus.cc:2258] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:43.865110 21992 ts_tablet_manager.cc:1905] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:43.867075 21992 ts_tablet_manager.cc:1918] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:43.867173 21992 log.cc:1196] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal/wals/55c03d51b5764abbaf5f3ad8360e1835
I20260809 00:45:43.867508 21992 ts_tablet_manager.cc:1939] T 55c03d51b5764abbaf5f3ad8360e1835 P 3806f5b0340e4d84888fe51a6c72ef70: Deleting consensus metadata
I20260809 00:45:43.868070 21677 catalog_manager.cc:4841] TS 3806f5b0340e4d84888fe51a6c72ef70 (127.30.104.193:45405): tablet 55c03d51b5764abbaf5f3ad8360e1835 (table test-table [id=fd7b5f8e4ed94f14843b924b0e43802b]) successfully deleted
W20260809 00:45:43.942162 21987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:43.942420 21987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:43.942448 21987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:43.943738 21987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:43.943789 21987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:43.943815 21987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:45:43.945335 21987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:44669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=42107
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:43.945540 21987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:43.945727 21987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:43.951473 22000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:43.951555 21999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:43.951643 21987 server_base.cc:1034] running on GCE node
W20260809 00:45:43.951488 22002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:43.952014 21987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:43.952292 21987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:43.953512 21987 hybrid_clock.cc:648] HybridClock initialized: now 1786236343953493 us; error 48 us; skew 500 ppm
Aug 09 00:45:43 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236343, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:43.956276 21987 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:45:43.956895 21987 webserver.cc:460] Webserver started at http://127.30.104.194:42107/ using document root <none> and password file <none>
I20260809 00:45:43.957190 21987 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:43.957288 21987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:43.958838 21987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:45:43.959677 22013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:43.960047 21987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:43.960170 21987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "669d9667d5d140eb8fef9cd5a07708f0"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "029aeafbe533300056d11d63461e8ec8"
server_key_iv: "eaf429e65c10608ee33bbfd7f99b7da5"
server_key_version: "encryptionkey@0"
I20260809 00:45:43.960587 21987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:43.972462 21987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:43.974968 21987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:43.975155 21987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:43.975489 21987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:43.975893 21987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:43.975945 21987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:43.975983 21987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:43.975997 21987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:43.981809 21987 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:44669
I20260809 00:45:43.981897 22178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:44669 every 8 connection(s)
I20260809 00:45:43.982256 21987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:45:43.982760 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 21987
I20260809 00:45:43.983095 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:42095
--local_ip_for_outbound_sockets=127.30.104.195
--tserver_master_addrs=127.30.104.254:37297
--webserver_port=41331
--webserver_interface=127.30.104.195
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 00:45:43 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236343, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:43.992139 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:43.983093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:40557 (local address 127.30.104.254:37297)
0809 00:45:43.983152 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:43.983155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:43.983455 (+ 300us) server_negotiation.cc:407] Connection header received
0809 00:45:43.984006 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:43.984007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:43.984024 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:43.984080 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:43.985075 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.985587 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.986606 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.986828 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.986923 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:43.989212 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:43.989214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:43.989214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:43.989222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:43.990658 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:43.991093 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:43.991094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:43.991095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:43.991163 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:43.991427 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:43.991429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:43.991429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:43.991574 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:43.991651 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:43.991944 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 00:45:43.992043 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:45:43.992393 22182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:43.983083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:40557)
0809 00:45:43.983294 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:43.983336 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:43.983894 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:43.984152 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:43.984156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:43.984764 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:43.984967 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:43.984972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.985690 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.985691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:43.986505 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:43.986508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:43.986936 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:43.986938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:43.987065 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:43.987723 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:43.987731 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:43.989109 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:43.990752 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:43.990760 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:43.990762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:43.990996 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:43.991233 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:43.991234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:43.991234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:43.991350 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:43.991656 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:43.991664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:43.991807 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 00:45:43.991920 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 00:45:43.992081 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
I20260809 00:45:43.992920 22180 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37297
I20260809 00:45:43.993046 22180 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:43.993281 22180 heartbeater.cc:511] Master 127.30.104.254:37297 requested a full tablet report, sending...
I20260809 00:45:43.993690 21697 ts_manager.cc:194] Registered new tserver with Master: 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669)
I20260809 00:45:43.994087 21697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:40557
W20260809 00:45:44.069496 22183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:45:44.069731 22183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:45:44.069806 22183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:45:44.071080 22183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:45:44.071125 22183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:45:44.071153 22183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:45:44.072641 22183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:42095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=41331
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:45:44.072928 22183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:45:44.073141 22183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:45:44.078504 22188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:44.078510 22191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:45:44.078511 22189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:45:44.078748 22183 server_base.cc:1034] running on GCE node
I20260809 00:45:44.079006 22183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:45:44.079272 22183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:45:44.080442 22183 hybrid_clock.cc:648] HybridClock initialized: now 1786236344080425 us; error 36 us; skew 500 ppm
Aug 09 00:45:44 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236344, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:45:44.083109 22183 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:45:44.083666 22183 webserver.cc:460] Webserver started at http://127.30.104.195:41331/ using document root <none> and password file <none>
I20260809 00:45:44.083902 22183 fs_manager.cc:359] Metadata directory not provided
I20260809 00:45:44.083986 22183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:45:44.085409 22183 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:44.086193 22201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:44.086421 22183 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:45:44.086555 22183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "1a8c339eb0e3eb48af685ee29b2d2619"
server_key_iv: "a2bb24bba2475d79b733f878da8fb20f"
server_key_version: "encryptionkey@0"
I20260809 00:45:44.086994 22183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:45:44.122948 22183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:45:44.125186 22183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:45:44.125356 22183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:45:44.125618 22183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:45:44.126013 22183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:45:44.126071 22183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:44.126116 22183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:45:44.126143 22183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:44.131961 22183 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:42095
I20260809 00:45:44.132006 22363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:42095 every 8 connection(s)
I20260809 00:45:44.132388 22183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
Aug 09 00:45:44 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236344, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:45:44.139720 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 22183
I20260809 00:45:44.142354 21982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.133198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:37377 (local address 127.30.104.254:37297)
0809 00:45:44.133288 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.133290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.133503 (+ 213us) server_negotiation.cc:407] Connection header received
0809 00:45:44.133984 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.133985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.133996 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.134060 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:44.135004 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.135585 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.136547 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.136737 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.136802 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.138981 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:44.138983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:44.138983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:44.138989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:44.140885 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:44.141297 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:44.141299 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:44.141299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:44.141367 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:44.141661 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:44.141663 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:44.141663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:44.141785 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:44.141866 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:44.142165 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 00:45:44.142257 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 00:45:44.142556 22367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.133169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:37377)
0809 00:45:44.133346 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:44.133389 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:44.133871 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:44.134112 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:44.134115 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:44.134716 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:45:44.134907 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.134912 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.135663 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.135663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.136456 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.136458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.136818 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.136818 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.136919 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.137441 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:45:44.137450 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:45:44.138894 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:45:44.140964 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:44.140973 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:44.140975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:44.141213 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:44.141455 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:45:44.141457 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:45:44.141457 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:45:44.141586 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:45:44.141857 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:45:44.141864 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:45:44.142026 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 00:45:44.142142 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 00:45:44.142314 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260809 00:45:44.143178 22364 heartbeater.cc:347] Connected to a master server at 127.30.104.254:37297
I20260809 00:45:44.143272 22364 heartbeater.cc:464] Registering TS with master...
I20260809 00:45:44.143527 22364 heartbeater.cc:511] Master 127.30.104.254:37297 requested a full tablet report, sending...
I20260809 00:45:44.144039 21697 ts_manager.cc:194] Registered new tserver with Master: 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
I20260809 00:45:44.144416 21697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:37377
I20260809 00:45:44.147305 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.140343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40012 (local address 127.30.104.254:37297)
0809 00:45:44.140496 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.140499 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.140578 (+ 79us) server_negotiation.cc:407] Connection header received
0809 00:45:44.140612 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.140612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.140627 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.140722 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:44.141338 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.141835 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.142698 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.143012 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.143085 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.144564 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:44.144566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:44.144566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:44.144574 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:44.146098 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:44.146436 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:44.146437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:44.146437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:44.146503 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:44.146789 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:44.146790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:44.146790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:44.146912 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:44.147055 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:44.147133 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 00:45:44.147203 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260809 00:45:44.148247 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:45:44.150843 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.148635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.30.104.254:37297)
0809 00:45:44.148708 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.148710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.148743 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:45:44.148804 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.148805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.148834 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.148899 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:45:44.149011 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.149484 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.150221 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.150427 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.150486 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.150633 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:45:44.150678 (+ 45us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 00:45:44.150798 (+ 120us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40028: token expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 00:45:44.150938 22370 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40028: token expired
W20260809 00:45:44.151186 21598 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.104.254:37297)
I20260809 00:45:44.158037 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.151495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40032 (local address 127.30.104.254:37297)
0809 00:45:44.151550 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.151551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.151583 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:45:44.151605 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.151605 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.151612 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.151648 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:45:44.152107 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.152564 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.153289 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.153482 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.153546 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.154793 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:45:44.154794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:45:44.154795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:45:44.154801 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:45:44.156531 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:44.156935 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:44.156936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:44.156936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:44.156997 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:45:44.157306 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:45:44.157307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:45:44.157307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:45:44.157492 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:45:44.157577 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:44.157797 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 00:45:44.157870 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:45:44.159227 21697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40032:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:45:44.160883 21697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:45:44.163393 21881 tablet_service.cc:1467] Processing CreateTablet for tablet e08629a350414c01ad027279e8d45fa8 (DEFAULT_TABLE table=test-table [id=174df82e232f4c80b810f84fa49ae2ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:44.163698 21881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e08629a350414c01ad027279e8d45fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:44.165390 21819 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Bootstrap starting.
I20260809 00:45:44.165740 21819 tablet_bootstrap.cc:654] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:44.166329 21819 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: No bootstrap required, opened a new log
I20260809 00:45:44.166380 21819 ts_tablet_manager.cc:1397] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 00:45:44.166494 21819 raft_consensus.cc:348] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.166558 21819 raft_consensus.cc:374] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:44.166599 21819 raft_consensus.cc:729] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3806f5b0340e4d84888fe51a6c72ef70, State: Initialized, Role: FOLLOWER
I20260809 00:45:44.166649 21819 consensus_queue.cc:260] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.166739 21819 ts_tablet_manager.cc:1428] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 00:45:44.167472 22372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.163136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:42095 (local address 127.0.0.1:38978)
0809 00:45:44.163302 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:44.163312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:44.163363 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:44.163845 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:44.163845 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:44.163847 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:44.163893 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.163894 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.165213 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.165214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.166290 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.166293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.167185 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.167186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.167265 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.167271 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:45:44.167303 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 00:45:44.167348 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260809 00:45:44.167568 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.163026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44669 (local address 127.0.0.1:38538)
0809 00:45:44.163103 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:44.163118 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:44.163163 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:44.163824 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:44.163826 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:44.163827 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:44.163881 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.163883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.164941 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.164942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.166023 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.166032 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.167299 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.167300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.167350 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.167352 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:45:44.167393 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 00:45:44.167463 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 00:45:44.167783 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.163376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38978 (local address 127.30.104.195:42095)
0809 00:45:44.163590 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.163595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.163606 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:45:44.163649 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.163652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.163668 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.163850 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:44.163947 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.165098 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.166379 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.167082 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.167581 (+ 499us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.167648 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:44.167675 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 00:45:44.167722 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":129,"threads_started":1}
I20260809 00:45:44.167775 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.163179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38538 (local address 127.30.104.194:44669)
0809 00:45:44.163633 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.163638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.163650 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:45:44.163691 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.163693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.163710 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.163971 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:44.164053 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.164859 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.166121 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.167134 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.167301 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.167373 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:44.167404 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 00:45:44.167664 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":97,"threads_started":1}
I20260809 00:45:44.168542 22272 tablet_service.cc:1467] Processing CreateTablet for tablet e08629a350414c01ad027279e8d45fa8 (DEFAULT_TABLE table=test-table [id=174df82e232f4c80b810f84fa49ae2ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:44.168624 22084 tablet_service.cc:1467] Processing CreateTablet for tablet e08629a350414c01ad027279e8d45fa8 (DEFAULT_TABLE table=test-table [id=174df82e232f4c80b810f84fa49ae2ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:45:44.168901 22272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e08629a350414c01ad027279e8d45fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:44.168938 22084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e08629a350414c01ad027279e8d45fa8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:45:44.172093 22379 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Bootstrap starting.
I20260809 00:45:44.172130 22380 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Bootstrap starting.
I20260809 00:45:44.172847 22379 tablet_bootstrap.cc:654] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:44.172868 22380 tablet_bootstrap.cc:654] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Neither blocks nor log segments found. Creating new log.
I20260809 00:45:44.173343 22379 log.cc:824] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:44.173375 22380 log.cc:824] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Log is configured to *not* fsync() on all Append() calls
I20260809 00:45:44.174165 22379 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: No bootstrap required, opened a new log
I20260809 00:45:44.174252 22379 ts_tablet_manager.cc:1397] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:45:44.174419 22380 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: No bootstrap required, opened a new log
I20260809 00:45:44.174477 22380 ts_tablet_manager.cc:1397] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:45:44.175784 22379 raft_consensus.cc:348] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.175971 22379 raft_consensus.cc:374] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:44.176014 22379 raft_consensus.cc:729] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 669d9667d5d140eb8fef9cd5a07708f0, State: Initialized, Role: FOLLOWER
I20260809 00:45:44.175935 22380 raft_consensus.cc:348] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.176189 22379 consensus_queue.cc:260] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.176282 22380 raft_consensus.cc:374] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:45:44.176376 22380 raft_consensus.cc:729] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d8fb51a1eca4e6883fe99a18e04e7ee, State: Initialized, Role: FOLLOWER
I20260809 00:45:44.176544 22379 ts_tablet_manager.cc:1428] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:45:44.176565 22180 heartbeater.cc:503] Master 127.30.104.254:37297 was elected leader, sending a full tablet report...
I20260809 00:45:44.176527 22380 consensus_queue.cc:260] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.176819 22380 ts_tablet_manager.cc:1428] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:45:44.176926 22364 heartbeater.cc:503] Master 127.30.104.254:37297 was elected leader, sending a full tablet report...
W20260809 00:45:44.235235 22181 tablet.cc:2369] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:45:44.337046 21980 tablet.cc:2369] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:45:44.383009 22365 tablet.cc:2369] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:45:44.551759 22388 raft_consensus.cc:482] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:45:44.551913 22388 raft_consensus.cc:504] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.552433 22388 leader_election.cc:288] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 3806f5b0340e4d84888fe51a6c72ef70 (127.30.104.193:45405)
I20260809 00:45:44.556986 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.552762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:53285 (local address 127.30.104.194:44669)
0809 00:45:44.552873 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.552875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.553005 (+ 130us) server_negotiation.cc:407] Connection header received
0809 00:45:44.553103 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.553105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.553117 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.553176 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:44.553384 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.554160 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.555907 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.556488 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.556588 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.556598 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:44.556857 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 00:45:44.556913 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 00:45:44.556999 22367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.552650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44669 (local address 127.30.104.195:53285)
0809 00:45:44.552919 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:44.552933 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:44.553009 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:44.553188 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:44.553188 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:44.553190 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:44.553291 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.553294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.554281 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.554285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.555789 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.555793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.556631 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.556632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.556761 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.556763 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:45:44.556830 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 00:45:44.556883 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":134}
I20260809 00:45:44.557044 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.552749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:37033 (local address 127.30.104.193:45405)
0809 00:45:44.552912 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.552914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.553090 (+ 176us) server_negotiation.cc:407] Connection header received
0809 00:45:44.553155 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.553156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.553168 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.553231 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:45:44.553385 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.554149 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.555908 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.556518 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.556600 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.556609 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:44.556905 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 00:45:44.556962 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260809 00:45:44.557008 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.552667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45405 (local address 127.30.104.195:37033)
0809 00:45:44.553020 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:44.553029 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:45:44.553083 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:45:44.553251 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:45:44.553252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:45:44.553253 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:45:44.553308 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.553310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.554280 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.554281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.555789 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:45:44.555793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.556631 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.556632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:45:44.556754 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.556759 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:45:44.556895 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 00:45:44.556928 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":290,"thread_start_us":91,"threads_started":1}
I20260809 00:45:44.557392 21911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08629a350414c01ad027279e8d45fa8" candidate_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3806f5b0340e4d84888fe51a6c72ef70" is_pre_election: true
I20260809 00:45:44.557472 22115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08629a350414c01ad027279e8d45fa8" candidate_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669d9667d5d140eb8fef9cd5a07708f0" is_pre_election: true
I20260809 00:45:44.557601 21911 raft_consensus.cc:2454] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d8fb51a1eca4e6883fe99a18e04e7ee in term 0.
I20260809 00:45:44.557601 22115 raft_consensus.cc:2454] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d8fb51a1eca4e6883fe99a18e04e7ee in term 0.
I20260809 00:45:44.557852 22206 leader_election.cc:302] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d8fb51a1eca4e6883fe99a18e04e7ee, 669d9667d5d140eb8fef9cd5a07708f0; no voters:
I20260809 00:45:44.558039 22388 raft_consensus.cc:2781] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:45:44.558115 22388 raft_consensus.cc:482] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:45:44.558167 22388 raft_consensus.cc:3037] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:44.558945 22388 raft_consensus.cc:504] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.559100 22388 leader_election.cc:288] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [CANDIDATE]: Term 1 election: Requested vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 3806f5b0340e4d84888fe51a6c72ef70 (127.30.104.193:45405)
I20260809 00:45:44.559271 21911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08629a350414c01ad027279e8d45fa8" candidate_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3806f5b0340e4d84888fe51a6c72ef70"
I20260809 00:45:44.559340 22115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e08629a350414c01ad027279e8d45fa8" candidate_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "669d9667d5d140eb8fef9cd5a07708f0"
I20260809 00:45:44.559386 21911 raft_consensus.cc:3037] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:44.559492 22115 raft_consensus.cc:3037] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:45:44.560122 21911 raft_consensus.cc:2454] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d8fb51a1eca4e6883fe99a18e04e7ee in term 1.
I20260809 00:45:44.560286 22115 raft_consensus.cc:2454] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d8fb51a1eca4e6883fe99a18e04e7ee in term 1.
I20260809 00:45:44.560381 22203 leader_election.cc:302] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d8fb51a1eca4e6883fe99a18e04e7ee, 3806f5b0340e4d84888fe51a6c72ef70; no voters:
I20260809 00:45:44.560472 22388 raft_consensus.cc:2781] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:45:44.560676 22388 raft_consensus.cc:686] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 LEADER]: Becoming Leader. State: Replica: 1d8fb51a1eca4e6883fe99a18e04e7ee, State: Running, Role: LEADER
I20260809 00:45:44.560761 22388 consensus_queue.cc:237] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:45:44.561899 21696 catalog_manager.cc:5495] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195). New cstate: current_term: 1 leader_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } health_report { overall_health: UNKNOWN } } }
I20260809 00:45:44.610725 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.608241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38990 (local address 127.30.104.195:42095)
0809 00:45:44.608310 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 00:45:44.608312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:45:44.608323 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:45:44.608371 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:45:44.608373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:45:44.608397 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:45:44.608479 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:45:44.608642 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.609154 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.609905 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:45:44.610115 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:45:44.610195 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:45:44.610375 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:45:44.610483 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:45:44.610582 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:45:44.610605 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 00:45:44.610629 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:45:44.612792 21911 raft_consensus.cc:1261] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Refusing update from remote peer 1d8fb51a1eca4e6883fe99a18e04e7ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:45:44.612807 22115 raft_consensus.cc:1261] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 1 FOLLOWER]: Refusing update from remote peer 1d8fb51a1eca4e6883fe99a18e04e7ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:45:44.613178 22388 consensus_queue.cc:1035] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:45:44.613353 22393 consensus_queue.cc:1035] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:45:44.618292 21696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:45:44.618403 21696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40032:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:45:44.619246 21696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Sending DeleteTablet for 3 replicas of tablet e08629a350414c01ad027279e8d45fa8
I20260809 00:45:44.619630 22272 tablet_service.cc:1514] Processing DeleteTablet for tablet e08629a350414c01ad027279e8d45fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:44 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38978
I20260809 00:45:44.619702 21881 tablet_service.cc:1514] Processing DeleteTablet for tablet e08629a350414c01ad027279e8d45fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:44 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:43474
I20260809 00:45:44.619722 22084 tablet_service.cc:1514] Processing DeleteTablet for tablet e08629a350414c01ad027279e8d45fa8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:45:44 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:38538
I20260809 00:45:44.620004 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 21756
I20260809 00:45:44.633291 22411 tablet_replica.cc:330] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: stopping tablet replica
I20260809 00:45:44.633483 22411 raft_consensus.cc:2229] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:45:44.633615 22411 raft_consensus.cc:2258] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:44.633888 22412 tablet_replica.cc:330] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: stopping tablet replica
I20260809 00:45:44.634032 22412 raft_consensus.cc:2229] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:45:44.634174 22412 raft_consensus.cc:2258] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:45:44.634192 22411 ts_tablet_manager.cc:1905] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:44.634706 22412 ts_tablet_manager.cc:1905] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:45:44.636202 22411 ts_tablet_manager.cc:1918] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:44.636343 22411 log.cc:1196] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/e08629a350414c01ad027279e8d45fa8
I20260809 00:45:44.636688 22411 ts_tablet_manager.cc:1939] T e08629a350414c01ad027279e8d45fa8 P 1d8fb51a1eca4e6883fe99a18e04e7ee: Deleting consensus metadata
I20260809 00:45:44.636965 22412 ts_tablet_manager.cc:1918] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:45:44.637120 22412 log.cc:1196] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/e08629a350414c01ad027279e8d45fa8
I20260809 00:45:44.637271 21680 catalog_manager.cc:4841] TS 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095): tablet e08629a350414c01ad027279e8d45fa8 (table test-table [id=174df82e232f4c80b810f84fa49ae2ce]) successfully deleted
I20260809 00:45:44.637531 22412 ts_tablet_manager.cc:1939] T e08629a350414c01ad027279e8d45fa8 P 669d9667d5d140eb8fef9cd5a07708f0: Deleting consensus metadata
I20260809 00:45:44.637954 21680 catalog_manager.cc:4841] TS 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669): tablet e08629a350414c01ad027279e8d45fa8 (table test-table [id=174df82e232f4c80b810f84fa49ae2ce]) successfully deleted
W20260809 00:45:44.639997 21677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:45405 (error 108)
I20260809 00:45:44.640415 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 21987
I20260809 00:45:44.640579 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:45:44.640389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:45405 (local address 127.0.0.1:43486)
0809 00:45:44.640476 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 00:45:44.640540 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:45405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 00:45:44.647697 21677 catalog_manager.cc:4584] TS 3806f5b0340e4d84888fe51a6c72ef70 (127.30.104.193:45405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e08629a350414c01ad027279e8d45fa8: Network error: Client connection negotiation failed: client connection to 127.30.104.193:45405: connect: Connection refused (error 111)
I20260809 00:45:44.651908 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 22183
I20260809 00:45:44.662066 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 21651
I20260809 00:46:04.674171 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37297
--webserver_interface=127.30.104.254
--webserver_port=37809
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:04.764582 22416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:04.764832 22416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:04.764863 22416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:04.766129 22416 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 00:46:04.766187 22416 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 00:46:04.766203 22416 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 00:46:04.766218 22416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:04.766233 22416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:04.766301 22416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:04.766321 22416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:04.766350 22416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:04.768069 22416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=37809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:04.768380 22416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:04.768744 22416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:04.774765 22424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:04.774881 22416 server_base.cc:1034] running on GCE node
W20260809 00:46:04.774755 22425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:04.774755 22428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:04.775342 22416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:04.775709 22416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:04.776896 22416 hybrid_clock.cc:648] HybridClock initialized: now 1786236364776873 us; error 41 us; skew 500 ppm
Aug 09 00:46:04 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236364, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:04.780069 22416 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:04.780678 22416 webserver.cc:460] Webserver started at http://127.30.104.254:37809/ using document root <none> and password file <none>
I20260809 00:46:04.780973 22416 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:04.781059 22416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:04.782676 22416 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:04.783648 22435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:04.783895 22416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:04.784019 22416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a476f00314fe4d93b40f60678a9c90cd"
format_stamp: "Formatted at 2026-08-09 00:45:22 on dist-test-slave-c12x"
server_key: "385d150bcd2a75e716d03bc513fc969f"
server_key_iv: "b5082e02688080edf3a219613d97abeb"
server_key_version: "encryptionkey@0"
I20260809 00:46:04.784413 22416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:04.792356 22416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:04.794783 22416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:04.794963 22416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:04.799527 22416 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37297
I20260809 00:46:04.799563 22504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37297 every 8 connection(s)
I20260809 00:46:04.799964 22416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:04.801206 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 22416
I20260809 00:46:04.801568 22505 sys_catalog.cc:263] Verifying existing consensus state
I20260809 00:46:04.801575 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:45405
--local_ip_for_outbound_sockets=127.30.104.193
--tserver_master_addrs=127.30.104.254:37297
--webserver_port=40299
--webserver_interface=127.30.104.193
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:46:04.802304 22505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Bootstrap starting.
I20260809 00:46:04.811672 22505 log.cc:824] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:04.814270 22505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:46:04.814657 22505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: Bootstrap complete.
I20260809 00:46:04.817363 22505 raft_consensus.cc:348] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:46:04.817677 22505 raft_consensus.cc:729] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a476f00314fe4d93b40f60678a9c90cd, State: Initialized, Role: FOLLOWER
I20260809 00:46:04.817870 22505 consensus_queue.cc:260] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:46:04.817953 22505 raft_consensus.cc:388] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:04.818009 22505 raft_consensus.cc:482] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:04.818092 22505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 2 FOLLOWER]: Advancing to term 3
I20260809 00:46:04.819144 22505 raft_consensus.cc:504] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:46:04.819272 22505 leader_election.cc:302] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a476f00314fe4d93b40f60678a9c90cd; no voters:
I20260809 00:46:04.819527 22505 leader_election.cc:288] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 00:46:04.819576 22509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 3 FOLLOWER]: Leader election won for term 3
I20260809 00:46:04.819736 22509 raft_consensus.cc:686] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [term 3 LEADER]: Becoming Leader. State: Replica: a476f00314fe4d93b40f60678a9c90cd, State: Running, Role: LEADER
I20260809 00:46:04.819832 22505 sys_catalog.cc:564] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:04.819895 22509 consensus_queue.cc:237] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } }
I20260809 00:46:04.820209 22510 sys_catalog.cc:455] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a476f00314fe4d93b40f60678a9c90cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } } }
I20260809 00:46:04.820207 22509 sys_catalog.cc:455] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a476f00314fe4d93b40f60678a9c90cd. Latest consensus state: current_term: 3 leader_uuid: "a476f00314fe4d93b40f60678a9c90cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a476f00314fe4d93b40f60678a9c90cd" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37297 } } }
I20260809 00:46:04.820363 22510 sys_catalog.cc:458] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:04.820402 22509 sys_catalog.cc:458] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:04.821847 22530 catalog_manager.cc:1259] Loaded cluster ID: 061f3052e5d346a9bf5cd014d47cdd63
I20260809 00:46:04.821902 22530 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: loading cluster ID for follower catalog manager: success
I20260809 00:46:04.823076 22530 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a476f00314fe4d93b40f60678a9c90cd: acquiring CA information for follower catalog manager: success
F20260809 00:46:04.823499 22530 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f41e4c329ad google::LogMessage::Fail() at ??:0
@ 0x7f41e4c3588a google::LogMessage::SendToLog() at ??:0
@ 0x7f41e4c324de google::LogMessage::Flush() at ??:0
@ 0x7f41e4c33a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f41e739f104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f41ecb8b6b9 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f41ecb8b989 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f41ecb8be4e kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f41ecb8e8c2 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f41e6af5954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f41e79596db start_thread at ??:0
@ 0x7f41e373061f clone at ??:0
W20260809 00:46:04.905356 22507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:04.905582 22507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:04.905608 22507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:04.906806 22507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:04.906849 22507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:04.906876 22507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:46:04.908414 22507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:45405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=40299
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:04.908692 22507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:04.908927 22507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:04.914342 22557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:04.914362 22559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:04.914351 22556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:04.915144 22507 server_base.cc:1034] running on GCE node
I20260809 00:46:04.915417 22507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:04.915813 22507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:04.917011 22507 hybrid_clock.cc:648] HybridClock initialized: now 1786236364916986 us; error 43 us; skew 500 ppm
Aug 09 00:46:04 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236364, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:04.920001 22507 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:46:04.920524 22507 webserver.cc:460] Webserver started at http://127.30.104.193:40299/ using document root <none> and password file <none>
I20260809 00:46:04.920804 22507 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:04.920881 22507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:04.922513 22507 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 00:46:04.923380 22567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:04.923619 22507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:04.923743 22507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "3806f5b0340e4d84888fe51a6c72ef70"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "4e26b744f82f11402d98437ea651a843"
server_key_iv: "87cfffeefecbb8b49dbf087655645e34"
server_key_version: "encryptionkey@0"
I20260809 00:46:04.924157 22507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:04.937291 22507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:04.939922 22507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:04.940122 22507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:04.940402 22507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:04.940925 22575 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 00:46:04.943027 22507 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 00:46:04.943115 22507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 00:46:04.943169 22507 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 00:46:04.944096 22507 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 00:46:04.944161 22507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 00:46:04.944263 22575 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Bootstrap starting.
I20260809 00:46:04.952703 22507 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:45405
I20260809 00:46:04.952888 22729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:45405 every 8 connection(s)
I20260809 00:46:04.953236 22507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:46:04.956067 22575 log.cc:824] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:04.958355 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 22507
I20260809 00:46:04.958719 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:44669
--local_ip_for_outbound_sockets=127.30.104.194
--tserver_master_addrs=127.30.104.254:37297
--webserver_port=42107
--webserver_interface=127.30.104.194
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:46:04.958938 22575 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 00:46:04.959292 22575 tablet_bootstrap.cc:492] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Bootstrap complete.
I20260809 00:46:04.960640 22575 ts_tablet_manager.cc:1397] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.012s
I20260809 00:46:04.962494 22575 raft_consensus.cc:348] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:46:04.962913 22575 raft_consensus.cc:729] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3806f5b0340e4d84888fe51a6c72ef70, State: Initialized, Role: FOLLOWER
I20260809 00:46:04.963080 22575 consensus_queue.cc:260] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:46:04.963398 22575 ts_tablet_manager.cc:1428] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260809 00:46:05.024747 22733 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:04.954081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:37045)
0809 00:46:04.954282 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:04.954331 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:04.954850 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:05.024518 (+ 69668us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
W20260809 00:46:05.025116 22731 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:37297 (0 consecutive failures): Network error: Failed to ping master at 127.30.104.254:37297: Client connection negotiation failed: client connection to 127.30.104.254:37297: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 00:46:05.025493 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:05.025363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:40259)
0809 00:46:05.025423 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:05.025471 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:46:05.025898 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:05.025801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:38357)
0809 00:46:05.025850 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:05.025882 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 00:46:05.026028 22731 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 00:46:05.048923 22735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:05.049160 22735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:05.049185 22735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:05.050348 22735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:05.050390 22735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:05.050416 22735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:46:05.051973 22735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:44669
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=42107
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22735
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:05.052232 22735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:05.052469 22735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:05.058333 22750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:05.058314 22743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:05.058468 22735 server_base.cc:1034] running on GCE node
W20260809 00:46:05.058314 22744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:05.058966 22735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:05.059233 22735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:05.060412 22735 hybrid_clock.cc:648] HybridClock initialized: now 1786236365060388 us; error 59 us; skew 500 ppm
Aug 09 00:46:05 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236365, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:05.063184 22735 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:46:05.063738 22735 webserver.cc:460] Webserver started at http://127.30.104.194:42107/ using document root <none> and password file <none>
I20260809 00:46:05.063992 22735 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:05.064064 22735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:05.065610 22735 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:46:05.066412 22759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:05.066659 22735 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:05.066779 22735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "669d9667d5d140eb8fef9cd5a07708f0"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "029aeafbe533300056d11d63461e8ec8"
server_key_iv: "eaf429e65c10608ee33bbfd7f99b7da5"
server_key_version: "encryptionkey@0"
I20260809 00:46:05.067195 22735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:05.086336 22735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:05.088927 22735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:05.089128 22735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:05.089392 22735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:05.089752 22735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:05.089818 22735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:05.089855 22735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:05.089910 22735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:05.097404 22735 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:44669
I20260809 00:46:05.097494 22920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:44669 every 8 connection(s)
I20260809 00:46:05.097975 22735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:46:05.099314 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:05.098807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:51913)
0809 00:46:05.099066 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:05.099149 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":134,"threads_started":1}
W20260809 00:46:05.100082 22921 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:37297 (0 consecutive failures): Network error: Failed to ping master at 127.30.104.254:37297: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
I20260809 00:46:05.100432 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:05.100314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:44557)
0809 00:46:05.100376 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:05.100410 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:46:05.100927 22927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:05.100802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:56903)
0809 00:46:05.100877 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:05.100907 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 00:46:05.101095 22921 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:46:05.106613 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 22735
I20260809 00:46:05.106937 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:42095
--local_ip_for_outbound_sockets=127.30.104.195
--tserver_master_addrs=127.30.104.254:37297
--webserver_port=41331
--webserver_interface=127.30.104.195
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:34139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:05.192766 22928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:05.193068 22928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:05.193095 22928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:05.194211 22928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:05.194252 22928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:05.194278 22928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:46:05.195744 22928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:34139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:42095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=41331
--tserver_master_addrs=127.30.104.254:37297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:05.195960 22928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:05.196142 22928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:05.201505 22934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:05.201529 22937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:05.201647 22933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:05.201749 22928 server_base.cc:1034] running on GCE node
I20260809 00:46:05.202011 22928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:05.202275 22928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:05.203510 22928 hybrid_clock.cc:648] HybridClock initialized: now 1786236365203497 us; error 28 us; skew 500 ppm
Aug 09 00:46:05 dist-test-slave-c12x krb5kdc[20885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236365, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:05.206064 22928 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:46:05.206559 22928 webserver.cc:460] Webserver started at http://127.30.104.195:41331/ using document root <none> and password file <none>
I20260809 00:46:05.206811 22928 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:05.206885 22928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:05.208453 22928 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:05.209173 22951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:05.209384 22928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:05.209540 22928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee"
format_stamp: "Formatted at 2026-08-09 00:45:23 on dist-test-slave-c12x"
server_key: "1a8c339eb0e3eb48af685ee29b2d2619"
server_key_iv: "a2bb24bba2475d79b733f878da8fb20f"
server_key_version: "encryptionkey@0"
I20260809 00:46:05.209929 22928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:05.224925 22928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:05.227919 22928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:05.228127 22928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:05.228427 22928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:05.228803 22928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:05.228870 22928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:05.228914 22928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:05.228945 22928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:05.235759 22928 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:42095
I20260809 00:46:05.235845 23118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:42095 every 8 connection(s)
I20260809 00:46:05.236245 22928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:46:05.237519 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:05.237022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:43661)
0809 00:46:05.237246 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:05.237351 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":81,"threads_started":1}
W20260809 00:46:05.237962 23119 heartbeater.cc:644] Failed to heartbeat to 127.30.104.254:37297 (0 consecutive failures): Network error: Failed to ping master at 127.30.104.254:37297: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
I20260809 00:46:05.238237 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:05.238144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:36183)
0809 00:46:05.238190 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:05.238219 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:46:05.238623 23121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:05.238483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:41337)
0809 00:46:05.238571 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:05.238602 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260809 00:46:05.238788 23119 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 00:46:05.244010 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 22928
I20260809 00:46:06.026649 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:06.026399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:36637)
0809 00:46:06.026539 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:06.026586 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260809 00:46:06.101711 23124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:06.101483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:52021)
0809 00:46:06.101612 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:06.101653 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":58,"threads_started":1}
I20260809 00:46:06.192965 23126 raft_consensus.cc:482] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:06.193149 23126 raft_consensus.cc:504] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:46:06.193650 23126 leader_election.cc:288] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
Aug 09 00:46:06 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236364, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.195@KRBTEST.COM
Aug 09 00:46:06 dist-test-slave-c12x krb5kdc[20885](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236364, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.194@KRBTEST.COM
I20260809 00:46:06.204877 23129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:06.194036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:33169 (local address 127.30.104.195:42095)
0809 00:46:06.194230 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:06.194238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:06.194254 (+ 16us) server_negotiation.cc:407] Connection header received
0809 00:46:06.194664 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:06.194666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:06.194714 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:06.194877 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:06.195748 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:06.196612 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:06.197685 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:06.198120 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:06.198260 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:06.201060 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:06.201065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:06.201065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:06.201097 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:06.202935 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:06.203373 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:06.203376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:06.203377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:06.203531 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:06.203868 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:06.203869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:06.203869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:06.204235 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:06.204331 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:06.204634 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 00:46:06.204767 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260809 00:46:06.204881 23127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:06.193923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:42095 (local address 127.30.104.193:33169)
0809 00:46:06.194061 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:06.194072 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:06.194128 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:06.194917 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:06.194920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:06.195335 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:06.195590 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:06.195596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:06.196744 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:06.196745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:06.197566 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:06.197569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:06.198214 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:06.198216 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:06.198318 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:06.199288 (+ 970us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:06.199294 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:06.200916 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:06.203041 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:06.203056 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:06.203058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:06.203284 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:06.203636 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:06.203636 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:06.203637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:06.203787 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:06.204319 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:06.204327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:06.204470 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 00:46:06.204580 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 00:46:06.204782 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":81,"thread_start_us":71,"threads_started":1}
I20260809 00:46:06.204979 23128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:06.193929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:46589 (local address 127.30.104.194:44669)
0809 00:46:06.194136 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:06.194148 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:06.194188 (+ 40us) server_negotiation.cc:407] Connection header received
0809 00:46:06.194660 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:06.194663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:06.194706 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:06.194876 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:06.195724 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:06.196618 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:06.197676 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:06.198090 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:06.198230 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:06.201485 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:06.201489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:06.201489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:06.201501 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:06.203274 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:06.203721 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:06.203725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:06.203725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:06.203874 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:06.204189 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:06.204189 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:06.204190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:06.204406 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:06.204503 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:06.204704 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 00:46:06.204865 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
W20260809 00:46:06.205806 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
I20260809 00:46:06.205978 23123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:06.193810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:44669 (local address 127.30.104.193:46589)
0809 00:46:06.193894 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:06.193920 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:06.194004 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:06.194917 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:06.194920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:06.195338 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:06.195590 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:06.195596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:06.196734 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:06.196735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:06.197566 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:06.197569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:06.198201 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:06.198202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:06.198320 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:06.199254 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:06.199274 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:06.201344 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:06.203394 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:06.203395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:06.203396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:06.203617 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:06.204062 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:06.204063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:06.204064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:06.204128 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:06.204487 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:06.204488 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:06.204537 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 00:46:06.204625 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 00:46:06.204871 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":162}
W20260809 00:46:06.206663 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
I20260809 00:46:06.206732 22572 leader_election.cc:302] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3806f5b0340e4d84888fe51a6c72ef70; no voters: 1d8fb51a1eca4e6883fe99a18e04e7ee, 669d9667d5d140eb8fef9cd5a07708f0
I20260809 00:46:06.206856 23126 raft_consensus.cc:2726] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:46:06.239473 23130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:06.239197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:49619)
0809 00:46:06.239334 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:06.239385 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260809 00:46:07.027561 23138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:07.027187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:57233)
0809 00:46:07.027365 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:07.027420 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":89,"threads_started":1}
I20260809 00:46:07.102614 23139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:07.102291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:59619)
0809 00:46:07.102523 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:07.102567 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":119,"threads_started":1}
I20260809 00:46:07.240244 23140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:07.240037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:56571)
0809 00:46:07.240163 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:07.240201 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260809 00:46:07.766561 23143 raft_consensus.cc:482] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:07.766670 23143 raft_consensus.cc:504] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:46:07.766875 23143 leader_election.cc:288] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
W20260809 00:46:07.767407 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
W20260809 00:46:07.767546 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
I20260809 00:46:07.767603 22572 leader_election.cc:302] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3806f5b0340e4d84888fe51a6c72ef70; no voters: 1d8fb51a1eca4e6883fe99a18e04e7ee, 669d9667d5d140eb8fef9cd5a07708f0
I20260809 00:46:07.767702 23143 raft_consensus.cc:2726] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:46:08.028465 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:08.028245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:39901)
0809 00:46:08.028377 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:08.028416 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":66,"threads_started":1}
I20260809 00:46:08.103684 23149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:08.103357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:54031)
0809 00:46:08.103570 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:08.103628 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":137,"threads_started":1}
I20260809 00:46:08.241135 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:08.240871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:42595)
0809 00:46:08.241042 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:08.241089 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":68,"threads_started":1}
I20260809 00:46:09.029388 23151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:09.029128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:56179)
0809 00:46:09.029291 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:09.029340 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260809 00:46:09.104611 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:09.104300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:38265)
0809 00:46:09.104514 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:09.104561 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
I20260809 00:46:09.242041 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:09.241783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:42813)
0809 00:46:09.241938 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:09.241985 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":75,"threads_started":1}
I20260809 00:46:09.475828 23155 raft_consensus.cc:482] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:09.475967 23155 raft_consensus.cc:504] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:46:09.476186 23155 leader_election.cc:288] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
W20260809 00:46:09.476718 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
W20260809 00:46:09.476816 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
I20260809 00:46:09.476855 22572 leader_election.cc:302] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3806f5b0340e4d84888fe51a6c72ef70; no voters: 1d8fb51a1eca4e6883fe99a18e04e7ee, 669d9667d5d140eb8fef9cd5a07708f0
I20260809 00:46:09.476972 23155 raft_consensus.cc:2726] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:46:10.030253 23156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:10.029987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:50829)
0809 00:46:10.030155 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:10.030205 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260809 00:46:10.105511 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:10.105222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:49659)
0809 00:46:10.105418 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:10.105462 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":93,"threads_started":1}
I20260809 00:46:10.242952 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:10.242674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:52389)
0809 00:46:10.242857 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:10.242903 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":101,"threads_started":1}
I20260809 00:46:11.031188 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:11.030881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:60337)
0809 00:46:11.031080 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:11.031137 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
I20260809 00:46:11.106477 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:11.106177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:43935)
0809 00:46:11.106366 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:11.106429 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":124,"threads_started":1}
I20260809 00:46:11.244028 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:11.243683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:38877)
0809 00:46:11.243880 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:11.243967 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":113,"threads_started":1}
I20260809 00:46:12.032153 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:12.031880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:50913)
0809 00:46:12.032051 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:12.032104 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260809 00:46:12.107357 23164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:12.107095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:53605)
0809 00:46:12.107263 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:12.107307 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260809 00:46:12.244901 23165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:12.244632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:41337)
0809 00:46:12.244806 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:12.244852 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260809 00:46:12.953994 23168 raft_consensus.cc:482] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:12.954104 23168 raft_consensus.cc:504] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:46:12.954263 23168 leader_election.cc:288] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
W20260809 00:46:12.954932 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
W20260809 00:46:12.955018 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
I20260809 00:46:12.955034 22572 leader_election.cc:302] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3806f5b0340e4d84888fe51a6c72ef70; no voters: 1d8fb51a1eca4e6883fe99a18e04e7ee, 669d9667d5d140eb8fef9cd5a07708f0
I20260809 00:46:12.955111 23168 raft_consensus.cc:2726] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:46:13.032984 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:13.032744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:55105)
0809 00:46:13.032888 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:13.032935 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":70,"threads_started":1}
I20260809 00:46:13.108080 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:13.107849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:36535)
0809 00:46:13.107981 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:13.108016 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260809 00:46:13.245847 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:13.245568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:51951)
0809 00:46:13.245737 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:13.245774 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260809 00:46:14.033859 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:14.033608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:55135)
0809 00:46:14.033763 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:14.033803 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260809 00:46:14.108767 23177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:14.108559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:32845)
0809 00:46:14.108683 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:14.108719 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260809 00:46:14.246569 23179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:14.246351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:39295)
0809 00:46:14.246482 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:14.246524 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260809 00:46:15.034838 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:15.034485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:55987)
0809 00:46:15.034734 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:15.034791 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
I20260809 00:46:15.109826 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:15.109434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:54091)
0809 00:46:15.109728 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:15.109777 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":153,"threads_started":1}
I20260809 00:46:15.247541 23184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:15.247179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:39349)
0809 00:46:15.247397 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:15.247434 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":116,"threads_started":1}
I20260809 00:46:16.035629 23185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:16.035334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:48759)
0809 00:46:16.035543 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:16.035581 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":80,"threads_started":1}
I20260809 00:46:16.110751 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:16.110473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:33127)
0809 00:46:16.110654 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:16.110702 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
I20260809 00:46:16.248629 23187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:16.248330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:48339)
0809 00:46:16.248528 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:16.248571 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":121,"threads_started":1}
I20260809 00:46:17.036491 23189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:17.036158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:58493)
0809 00:46:17.036387 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:17.036440 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260809 00:46:17.111889 23190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:17.111410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:34217)
0809 00:46:17.111780 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:17.111836 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":168,"threads_started":1}
I20260809 00:46:17.249526 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:17.249237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:33523)
0809 00:46:17.249421 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:17.249475 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":91,"threads_started":1}
I20260809 00:46:18.037530 23192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:18.037179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:58705)
0809 00:46:18.037420 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:18.037481 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":80,"threads_started":1}
I20260809 00:46:18.112841 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:18.112510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:44399)
0809 00:46:18.112731 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:18.112789 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":135,"threads_started":1}
I20260809 00:46:18.250545 23194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:18.250138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:35661)
0809 00:46:18.250456 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:18.250495 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":232,"threads_started":1}
I20260809 00:46:19.038511 23195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:19.038205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:50919)
0809 00:46:19.038385 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:19.038462 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260809 00:46:19.113729 23197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:19.113434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:55085)
0809 00:46:19.113597 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:19.113662 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260809 00:46:19.251504 23198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:19.251193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:39209)
0809 00:46:19.251352 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:19.251452 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260809 00:46:19.960400 23199 raft_consensus.cc:482] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:19.960526 23199 raft_consensus.cc:504] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "669d9667d5d140eb8fef9cd5a07708f0" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 44669 } } peers { permanent_uuid: "1d8fb51a1eca4e6883fe99a18e04e7ee" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 42095 } } peers { permanent_uuid: "3806f5b0340e4d84888fe51a6c72ef70" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 45405 } }
I20260809 00:46:19.960727 23199 leader_election.cc:288] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669), 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095)
W20260809 00:46:19.961246 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 669d9667d5d140eb8fef9cd5a07708f0 (127.30.104.194:44669): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
W20260809 00:46:19.961350 22572 leader_election.cc:341] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1d8fb51a1eca4e6883fe99a18e04e7ee (127.30.104.195:42095): Not found: Tablet not found: e08629a350414c01ad027279e8d45fa8
I20260809 00:46:19.961402 22572 leader_election.cc:302] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3806f5b0340e4d84888fe51a6c72ef70; no voters: 1d8fb51a1eca4e6883fe99a18e04e7ee, 669d9667d5d140eb8fef9cd5a07708f0
I20260809 00:46:19.961488 23199 raft_consensus.cc:2726] T e08629a350414c01ad027279e8d45fa8 P 3806f5b0340e4d84888fe51a6c72ef70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 00:46:20.039656 23200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:20.039280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.193:37771)
0809 00:46:20.039548 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:20.039603 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":75,"threads_started":1}
I20260809 00:46:20.114655 23201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:20.114369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.194:47775)
0809 00:46:20.114555 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:20.114604 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":88,"threads_started":1}
I20260809 00:46:20.245565 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 22507
I20260809 00:46:20.252424 23202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:20.252168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:37297 (local address 127.30.104.195:41741)
0809 00:46:20.252336 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:20.252375 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.254:37297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260809 00:46:20.255089 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 22735
I20260809 00:46:20.267222 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 22928
2026-08-09T00:46:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58930 ms)
[----------] 30 tests from SecurityITest (249085 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:20 dist-test-slave-c12x krb5kdc[23206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:20 dist-test-slave-c12x krb5kdc[23206](info): set up 2 sockets
Aug 09 00:46:20 dist-test-slave-c12x krb5kdc[23206](info): commencing operation
krb5kdc: starting...
W20260809 00:46:21.518035 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:21 dist-test-slave-c12x krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:21.655714 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39179
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:39179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:21.744270 23225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:21.744496 23225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:21.744521 23225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:21.745762 23225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:21.745813 23225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:21.745828 23225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:21.745841 23225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:21.745853 23225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:21.747547 23225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:39179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:21.747843 23225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:21.748108 23225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:21.753369 23232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:21.753592 23225 server_base.cc:1034] running on GCE node
W20260809 00:46:21.753335 23235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:21.753337 23231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:21.753937 23225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:21.754297 23225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:21.755512 23225 hybrid_clock.cc:648] HybridClock initialized: now 1786236381755487 us; error 41 us; skew 500 ppm
Aug 09 00:46:21 dist-test-slave-c12x krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236381, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:21.758301 23225 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:21.758862 23225 webserver.cc:460] Webserver started at http://127.30.104.254:43099/ using document root <none> and password file <none>
I20260809 00:46:21.759133 23225 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:21.759182 23225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:21.759328 23225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:21.760524 23225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "8d44d66a0415467f93a9c924c622f66c"
format_stamp: "Formatted at 2026-08-09 00:46:21 on dist-test-slave-c12x"
server_key: "cc4961f3b29dfd02f8c81fcab81d3be7"
server_key_iv: "f23b429284605ac9b37a72c8e4b11694"
server_key_version: "encryptionkey@0"
I20260809 00:46:21.760918 23225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "8d44d66a0415467f93a9c924c622f66c"
format_stamp: "Formatted at 2026-08-09 00:46:21 on dist-test-slave-c12x"
server_key: "cc4961f3b29dfd02f8c81fcab81d3be7"
server_key_iv: "f23b429284605ac9b37a72c8e4b11694"
server_key_version: "encryptionkey@0"
I20260809 00:46:21.762379 23225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:46:21.763221 23246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:21.763420 23225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:21.763594 23225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "8d44d66a0415467f93a9c924c622f66c"
format_stamp: "Formatted at 2026-08-09 00:46:21 on dist-test-slave-c12x"
server_key: "cc4961f3b29dfd02f8c81fcab81d3be7"
server_key_iv: "f23b429284605ac9b37a72c8e4b11694"
server_key_version: "encryptionkey@0"
I20260809 00:46:21.763684 23225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:21.775974 23225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:21.778652 23225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:21.778836 23225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:21.783795 23225 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:39179
I20260809 00:46:21.783836 23325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:39179 every 8 connection(s)
I20260809 00:46:21.784250 23225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:21.785284 23326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:21.787938 23326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c: Bootstrap starting.
I20260809 00:46:21.788575 23326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:21.788898 23326 log.cc:824] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:21.789755 23326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c: No bootstrap required, opened a new log
I20260809 00:46:21.791524 23326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d44d66a0415467f93a9c924c622f66c" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39179 } }
I20260809 00:46:21.791689 23326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:21.791714 23326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d44d66a0415467f93a9c924c622f66c, State: Initialized, Role: FOLLOWER
I20260809 00:46:21.791875 23326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d44d66a0415467f93a9c924c622f66c" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39179 } }
I20260809 00:46:21.791957 23326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:21.791980 23326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:21.792032 23326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:21.792479 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 23225
I20260809 00:46:21.792595 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:21.792811 31139 external_mini_cluster.cc:1428] Setting key e6634bd998b7d728d2e235e0923711cd
I20260809 00:46:21.792860 23326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d44d66a0415467f93a9c924c622f66c" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39179 } }
I20260809 00:46:21.793000 23326 leader_election.cc:302] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d44d66a0415467f93a9c924c622f66c; no voters:
I20260809 00:46:21.793205 23326 leader_election.cc:288] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:21.793512 23326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:21.793517 23329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d44d66a0415467f93a9c924c622f66c" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39179 } } }
I20260809 00:46:21.793661 23329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:46:21.793977 23337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:21.794189 23337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [term 1 LEADER]: Becoming Leader. State: Replica: 8d44d66a0415467f93a9c924c622f66c, State: Running, Role: LEADER
I20260809 00:46:21.794376 23337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d44d66a0415467f93a9c924c622f66c" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39179 } }
I20260809 00:46:21.794816 23329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d44d66a0415467f93a9c924c622f66c. Latest consensus state: current_term: 1 leader_uuid: "8d44d66a0415467f93a9c924c622f66c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d44d66a0415467f93a9c924c622f66c" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39179 } } }
I20260809 00:46:21.794895 23329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:21.795094 23357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:21.795820 23357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:21.798136 23357 catalog_manager.cc:1347] Generated new cluster ID: a421cf1a4df3404b9b7de5a461061178
I20260809 00:46:21.798211 23357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:46:21 dist-test-slave-c12x krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:21.805837 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:21.793562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43994 (local address 127.30.104.254:39179)
0809 00:46:21.793832 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:21.793845 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:21.793883 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:46:21.794011 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:21.794019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:21.794085 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:21.795171 (+ 1086us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:21.795416 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:21.796900 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:21.797877 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:21.798538 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:21.798651 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:21.801903 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:21.801907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:21.801908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:21.801922 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:21.804040 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:21.804485 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:21.804488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:21.804488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:21.804612 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:21.804844 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:21.804845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:21.804846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:21.805160 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:21.805320 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:21.805462 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 00:46:21.805602 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":182,"threads_started":1}
I20260809 00:46:21.822847 23357 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:21.823566 23357 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:21.832952 23357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c: Generated new TSK 0
I20260809 00:46:21.833216 23357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:46:21.871848 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39179
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:21.958513 23367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:21.958745 23367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:21.958770 23367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:21.960038 23367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:21.960083 23367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:21.960109 23367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:46:21.961627 23367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:21.961905 23367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:21.962178 23367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:21.968267 23375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:21.968248 23378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:21.968506 23367 server_base.cc:1034] running on GCE node
W20260809 00:46:21.968248 23374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:21.968788 23367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:21.969170 23367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:21.970534 23367 hybrid_clock.cc:648] HybridClock initialized: now 1786236381970514 us; error 32 us; skew 500 ppm
Aug 09 00:46:21 dist-test-slave-c12x krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236381, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:21.973362 23367 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:46:21.973863 23367 webserver.cc:460] Webserver started at http://127.30.104.193:43413/ using document root <none> and password file <none>
I20260809 00:46:21.974109 23367 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:21.974159 23367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:21.974299 23367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:21.975407 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "cbf4d36649a24bba9bbabe77dbe954a2"
format_stamp: "Formatted at 2026-08-09 00:46:21 on dist-test-slave-c12x"
server_key: "f14cf7cc722e3c12b3435bcf28dc9fd0"
server_key_iv: "84419f1e8a2c8ff8e8f194975a48b6c1"
server_key_version: "encryptionkey@0"
I20260809 00:46:21.975869 23367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "cbf4d36649a24bba9bbabe77dbe954a2"
format_stamp: "Formatted at 2026-08-09 00:46:21 on dist-test-slave-c12x"
server_key: "f14cf7cc722e3c12b3435bcf28dc9fd0"
server_key_iv: "84419f1e8a2c8ff8e8f194975a48b6c1"
server_key_version: "encryptionkey@0"
I20260809 00:46:21.977295 23367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:46:21.978134 23391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:21.978369 23367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:21.978499 23367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "cbf4d36649a24bba9bbabe77dbe954a2"
format_stamp: "Formatted at 2026-08-09 00:46:21 on dist-test-slave-c12x"
server_key: "f14cf7cc722e3c12b3435bcf28dc9fd0"
server_key_iv: "84419f1e8a2c8ff8e8f194975a48b6c1"
server_key_version: "encryptionkey@0"
I20260809 00:46:21.978580 23367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:21.994451 23367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:21.997201 23367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:21.997380 23367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:21.997691 23367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:21.998059 23367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:21.998095 23367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:21.998118 23367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:21.998154 23367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.004676 23367 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:37211
I20260809 00:46:22.004763 23551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:37211 every 8 connection(s)
I20260809 00:46:22.005146 23367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:46:22.009562 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 23367
I20260809 00:46:22.009697 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:46:22.009909 31139 external_mini_cluster.cc:1428] Setting key db66dde658041638996971e502f6b5fa
Aug 09 00:46:22 dist-test-slave-c12x krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236381, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:22.015915 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.005971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:50801 (local address 127.30.104.254:39179)
0809 00:46:22.006018 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.006020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.006242 (+ 222us) server_negotiation.cc:407] Connection header received
0809 00:46:22.006775 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.006777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.006792 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.006865 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:22.007919 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.008436 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.009730 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.010002 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.010096 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.012627 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:22.012629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:22.012629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:22.012637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:22.014290 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:22.014785 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:22.014787 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:22.014787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:22.014854 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:22.015141 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:22.015142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:22.015142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:22.015264 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:22.015343 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.015709 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.015816 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:46:22.016158 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.005909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39179 (local address 127.30.104.193:50801)
0809 00:46:22.006106 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.006151 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.006651 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.006947 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.006951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.007610 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:22.007817 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.007822 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.008593 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.008594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.009614 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.009618 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.010108 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.010110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.010298 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.010973 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:22.010982 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:22.012520 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:22.014420 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:22.014428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:22.014431 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:22.014701 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:22.014930 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:22.014931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:22.014931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:22.015059 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:22.015335 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:22.015343 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:22.015564 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.015686 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.015872 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":113,"threads_started":1}
I20260809 00:46:22.016978 23553 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39179
I20260809 00:46:22.017105 23553 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:22.017303 23553 heartbeater.cc:511] Master 127.30.104.254:39179 requested a full tablet report, sending...
I20260809 00:46:22.017942 23271 ts_manager.cc:194] Registered new tserver with Master: cbf4d36649a24bba9bbabe77dbe954a2 (127.30.104.193:37211)
I20260809 00:46:22.018940 23271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:50801
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:46:22.042527 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39179
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:22.135661 23559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:22.135893 23559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:22.135918 23559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:22.137074 23559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:22.137112 23559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:22.137140 23559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:46:22.138638 23559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:22.138862 23559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:22.139060 23559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:46:22.144593 23559 server_base.cc:1034] running on GCE node
W20260809 00:46:22.144580 23566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.144574 23568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.144553 23565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:22.145121 23559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:22.145404 23559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:22.146608 23559 hybrid_clock.cc:648] HybridClock initialized: now 1786236382146594 us; error 46 us; skew 500 ppm
Aug 09 00:46:22 dist-test-slave-c12x krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236382, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:22.149575 23559 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:46:22.150157 23559 webserver.cc:460] Webserver started at http://127.30.104.194:36517/ using document root <none> and password file <none>
I20260809 00:46:22.150413 23559 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:22.150506 23559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:22.150673 23559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:22.151993 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "f8d4c5d20e3b42b6bd66f8be283c3186"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "50a3e6a13efca939f23e5e15f235f16e"
server_key_iv: "9206aa89803c767aec7cb004f59743eb"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.152420 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "f8d4c5d20e3b42b6bd66f8be283c3186"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "50a3e6a13efca939f23e5e15f235f16e"
server_key_iv: "9206aa89803c767aec7cb004f59743eb"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.154022 23559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:46:22.155002 23576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.155355 23559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:22.155521 23559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "f8d4c5d20e3b42b6bd66f8be283c3186"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "50a3e6a13efca939f23e5e15f235f16e"
server_key_iv: "9206aa89803c767aec7cb004f59743eb"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.155611 23559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:22.174602 23559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:22.177314 23559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:22.177505 23559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:22.177783 23559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:22.178134 23559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:22.178194 23559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.178242 23559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:22.178267 23559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.185186 23559 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:39909
I20260809 00:46:22.185226 23741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:39909 every 8 connection(s)
I20260809 00:46:22.185587 23559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:46:22.189648 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 23559
I20260809 00:46:22.189810 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:46:22.190035 31139 external_mini_cluster.cc:1428] Setting key 7a89cc8b14d68313d814743fd81fdb44
Aug 09 00:46:22 dist-test-slave-c12x krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236382, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:22.196859 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.186418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:34643 (local address 127.30.104.254:39179)
0809 00:46:22.186498 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.186500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.186663 (+ 163us) server_negotiation.cc:407] Connection header received
0809 00:46:22.187188 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.187190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.187201 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.187269 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:22.188324 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.188872 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.189908 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.190177 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.190278 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.193295 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:22.193297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:22.193298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:22.193308 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:22.195064 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:22.195703 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:22.195704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:22.195705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:22.195768 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:22.196129 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:22.196129 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:22.196129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:22.196243 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:22.196296 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.196702 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.196788 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:46:22.197129 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.186335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39179 (local address 127.30.104.194:34643)
0809 00:46:22.186511 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.186556 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.187056 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.187325 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.187328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.188008 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:22.188215 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.188220 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.188967 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.188968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.189778 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.189780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.190293 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.190294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.190460 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.191117 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:22.191126 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:22.193184 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:22.195156 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:22.195164 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:22.195166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:22.195589 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:22.195857 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:22.195858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:22.195858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:22.196030 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:22.196313 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:22.196322 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:22.196542 (+ 220us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.196662 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.196862 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260809 00:46:22.197662 23742 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39179
I20260809 00:46:22.197786 23742 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:22.198019 23742 heartbeater.cc:511] Master 127.30.104.254:39179 requested a full tablet report, sending...
I20260809 00:46:22.198504 23271 ts_manager.cc:194] Registered new tserver with Master: f8d4c5d20e3b42b6bd66f8be283c3186 (127.30.104.194:39909)
I20260809 00:46:22.198925 23271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:34643
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:46:22.224067 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39179
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:22.315634 23752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:22.315869 23752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:22.315894 23752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:22.317114 23752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:22.317162 23752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:22.317190 23752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:46:22.318758 23752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:39179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:22.319016 23752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:22.319272 23752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:46:22.325313 23752 server_base.cc:1034] running on GCE node
W20260809 00:46:22.325248 23760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.325249 23764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.325253 23761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:22.325773 23752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:22.326038 23752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:22.327204 23752 hybrid_clock.cc:648] HybridClock initialized: now 1786236382327190 us; error 30 us; skew 500 ppm
Aug 09 00:46:22 dist-test-slave-c12x krb5kdc[23206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236382, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:22.329860 23752 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:46:22.330336 23752 webserver.cc:460] Webserver started at http://127.30.104.195:33095/ using document root <none> and password file <none>
I20260809 00:46:22.330578 23752 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:22.330627 23752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:22.330792 23752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:22.331979 23752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "5e5d3df756894acd8e8f5cd471c80468"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "9f44188c9dd92cc27711819a3ef98ff0"
server_key_iv: "41129b3e5710244fa0d83d140ec75726"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.332373 23752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "5e5d3df756894acd8e8f5cd471c80468"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "9f44188c9dd92cc27711819a3ef98ff0"
server_key_iv: "41129b3e5710244fa0d83d140ec75726"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.333847 23752 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:46:22.334643 23773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.334878 23752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:22.335009 23752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "5e5d3df756894acd8e8f5cd471c80468"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "9f44188c9dd92cc27711819a3ef98ff0"
server_key_iv: "41129b3e5710244fa0d83d140ec75726"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.335093 23752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:22.348453 23752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:22.350603 23752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:22.350786 23752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:22.351051 23752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:22.351390 23752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:22.351475 23752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.351523 23752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:22.351557 23752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.357981 23752 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:34521
I20260809 00:46:22.358039 23944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:34521 every 8 connection(s)
I20260809 00:46:22.358371 23752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:46:22.361109 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 23752
I20260809 00:46:22.361230 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:46:22.361402 31139 external_mini_cluster.cc:1428] Setting key b56e32a6b7f306e85d3babb014d3a5da
Aug 09 00:46:22 dist-test-slave-c12x krb5kdc[23206](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236382, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:22.368613 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.359192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:37505 (local address 127.30.104.254:39179)
0809 00:46:22.359278 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.359281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.359513 (+ 232us) server_negotiation.cc:407] Connection header received
0809 00:46:22.360038 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.360039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.360050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.360110 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:22.361246 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.361869 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.363033 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.363237 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.363317 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.365609 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:22.365611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:22.365611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:22.365618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:22.367061 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:22.367569 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:22.367570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:22.367570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:22.367636 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:22.367940 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:22.367941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:22.367941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:22.368054 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:22.368118 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.368447 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.368543 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:46:22.368791 23948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.359118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:39179 (local address 127.30.104.195:37505)
0809 00:46:22.359330 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.359369 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.359926 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.360173 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.360176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.360775 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:22.361098 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.361105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.361992 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.361993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.362939 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.362949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.363344 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.363346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.363554 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.364063 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:22.364071 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:22.365497 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:22.367160 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:22.367166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:22.367168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:22.367407 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:22.367743 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:22.367744 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:22.367745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:22.367863 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:22.368122 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:22.368128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:22.368265 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.368358 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.368543 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":134,"threads_started":1}
I20260809 00:46:22.369427 23945 heartbeater.cc:347] Connected to a master server at 127.30.104.254:39179
I20260809 00:46:22.369652 23945 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:22.369927 23945 heartbeater.cc:511] Master 127.30.104.254:39179 requested a full tablet report, sending...
I20260809 00:46:22.370385 23271 ts_manager.cc:194] Registered new tserver with Master: 5e5d3df756894acd8e8f5cd471c80468 (127.30.104.195:34521)
I20260809 00:46:22.370977 23271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:37505
I20260809 00:46:22.373272 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:46:22.381729 23338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.374666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43998 (local address 127.30.104.254:39179)
0809 00:46:22.374744 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.374746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.374866 (+ 120us) server_negotiation.cc:407] Connection header received
0809 00:46:22.374981 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.374982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.374993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.375045 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:22.375700 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.376190 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.377108 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.377458 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.377521 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.378854 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:22.378856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:22.378856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:22.378864 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:22.380437 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:22.380805 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:22.380806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:22.380807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:22.380872 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:22.381100 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:22.381100 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:22.381101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:22.381311 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:22.381426 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.381539 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.381624 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:46:22.383757 23271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:46:22.385382 23271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:46:22.392647 23965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.388235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:34521 (local address 127.0.0.1:59570)
0809 00:46:22.388608 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.388654 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.388728 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.389033 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.389037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.389039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:22.389135 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.389137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.390115 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.390116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.391298 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.391301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.392295 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.392296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.392379 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.392393 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.392471 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.392541 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"spinlock_wait_cycles":768,"thread_start_us":108,"threads_started":1}
I20260809 00:46:22.393031 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.388369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.30.104.195:34521)
0809 00:46:22.388837 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.388841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.388853 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:46:22.388895 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.388897 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.388913 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.389015 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:22.389218 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.390009 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.391404 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.392201 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.392280 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.392340 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.392881 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.392940 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":125,"threads_started":1}
I20260809 00:46:22.392999 23968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.388235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:39909 (local address 127.0.0.1:49534)
0809 00:46:22.388642 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.388654 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.388793 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.389275 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.389275 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.389277 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:22.389318 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.389319 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.390636 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.390637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.391762 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.391765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.392586 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.392587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.392657 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.392663 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.392844 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.392889 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":4,"thread_start_us":65,"threads_started":1}
I20260809 00:46:22.393327 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.388510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:37211 (local address 127.0.0.1:55538)
0809 00:46:22.388644 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.388654 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.388728 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.389651 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.389652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.389654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:22.389695 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.389697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.390934 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.390935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.391856 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.391859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.392852 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.392853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.392908 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.392911 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.392963 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.392996 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260809 00:46:22.393532 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.388616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55538 (local address 127.30.104.193:37211)
0809 00:46:22.389410 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.389414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.389423 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:46:22.389463 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.389465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.389478 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.389554 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:22.389772 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.390818 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.391932 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.392689 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.392755 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.392814 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.393082 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.393136 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":92,"threads_started":1}
I20260809 00:46:22.393857 23966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.388370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49534 (local address 127.30.104.194:39909)
0809 00:46:22.389159 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.389162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.389170 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:46:22.389200 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.389201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.389212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.389272 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:22.389407 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.390523 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.391851 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.392506 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.392572 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.392623 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.392760 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.393763 (+ 1003us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":118,"threads_started":1}
I20260809 00:46:22.393811 23850 tablet_service.cc:1467] Processing CreateTablet for tablet 0da710fa023549219e796b48477afd64 (DEFAULT_TABLE table=test-table [id=b7bde5aeb06e4542b3324f1bfb8db837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:22.394269 23850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0da710fa023549219e796b48477afd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:22.394241 23451 tablet_service.cc:1467] Processing CreateTablet for tablet 0da710fa023549219e796b48477afd64 (DEFAULT_TABLE table=test-table [id=b7bde5aeb06e4542b3324f1bfb8db837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:22.394579 23451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0da710fa023549219e796b48477afd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:22.396083 23640 tablet_service.cc:1467] Processing CreateTablet for tablet 0da710fa023549219e796b48477afd64 (DEFAULT_TABLE table=test-table [id=b7bde5aeb06e4542b3324f1bfb8db837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:22.396416 23640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0da710fa023549219e796b48477afd64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:22.397694 23972 tablet_bootstrap.cc:492] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2: Bootstrap starting.
I20260809 00:46:22.397912 23973 tablet_bootstrap.cc:492] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: Bootstrap starting.
I20260809 00:46:22.398576 23973 tablet_bootstrap.cc:654] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:22.398576 23972 tablet_bootstrap.cc:654] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:22.399243 23972 log.cc:824] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:22.399243 23973 log.cc:824] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:22.400112 23976 tablet_bootstrap.cc:492] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Bootstrap starting.
I20260809 00:46:22.400166 23973 tablet_bootstrap.cc:492] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: No bootstrap required, opened a new log
I20260809 00:46:22.400219 23973 ts_tablet_manager.cc:1397] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:46:22.400645 23972 tablet_bootstrap.cc:492] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2: No bootstrap required, opened a new log
I20260809 00:46:22.400780 23972 ts_tablet_manager.cc:1397] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 00:46:22.400825 23976 tablet_bootstrap.cc:654] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:22.401441 23976 log.cc:824] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:22.401571 23973 raft_consensus.cc:348] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.401770 23973 raft_consensus.cc:374] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:22.401821 23973 raft_consensus.cc:729] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5d3df756894acd8e8f5cd471c80468, State: Initialized, Role: FOLLOWER
I20260809 00:46:22.401958 23973 consensus_queue.cc:260] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.402276 23973 ts_tablet_manager.cc:1428] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:46:22.402305 23976 tablet_bootstrap.cc:492] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: No bootstrap required, opened a new log
I20260809 00:46:22.402370 23976 ts_tablet_manager.cc:1397] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:46:22.402508 23945 heartbeater.cc:503] Master 127.30.104.254:39179 was elected leader, sending a full tablet report...
I20260809 00:46:22.403163 23972 raft_consensus.cc:348] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.403354 23972 raft_consensus.cc:374] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:22.403400 23972 raft_consensus.cc:729] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf4d36649a24bba9bbabe77dbe954a2, State: Initialized, Role: FOLLOWER
I20260809 00:46:22.403573 23972 consensus_queue.cc:260] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.403923 23553 heartbeater.cc:503] Master 127.30.104.254:39179 was elected leader, sending a full tablet report...
I20260809 00:46:22.403975 23972 ts_tablet_manager.cc:1428] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:46:22.404430 23976 raft_consensus.cc:348] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.404608 23976 raft_consensus.cc:374] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:22.404662 23976 raft_consensus.cc:729] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8d4c5d20e3b42b6bd66f8be283c3186, State: Initialized, Role: FOLLOWER
I20260809 00:46:22.404783 23976 consensus_queue.cc:260] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.405093 23976 ts_tablet_manager.cc:1428] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:46:22.405139 23742 heartbeater.cc:503] Master 127.30.104.254:39179 was elected leader, sending a full tablet report...
I20260809 00:46:22.416493 23981 raft_consensus.cc:482] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:22.416628 23981 raft_consensus.cc:504] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.417135 23981 leader_election.cc:288] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e5d3df756894acd8e8f5cd471c80468 (127.30.104.195:34521), cbf4d36649a24bba9bbabe77dbe954a2 (127.30.104.193:37211)
I20260809 00:46:22.421200 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.417436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:60177 (local address 127.30.104.193:37211)
0809 00:46:22.417507 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.417510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.417521 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:46:22.417553 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.417554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.417565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.417635 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:22.417822 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.418620 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.420235 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.420824 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.420897 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.420906 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.421094 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.421131 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 00:46:22.421200 23967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.417519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:34795 (local address 127.30.104.195:34521)
0809 00:46:22.417566 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.417568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.417705 (+ 137us) server_negotiation.cc:407] Connection header received
0809 00:46:22.417755 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.417756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.417764 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.417836 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:22.417964 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.418642 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.420235 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.420813 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.420897 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.420906 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.421081 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.421124 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:46:22.421254 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.417333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:37211 (local address 127.30.104.194:60177)
0809 00:46:22.417420 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.417434 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.417483 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.417653 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.417654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.417657 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:22.417745 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.417747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.418748 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.418748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.420141 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.420145 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.420916 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.420917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.421002 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.421004 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.421066 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.421121 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":83}
I20260809 00:46:22.421454 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.417338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:34521 (local address 127.30.104.194:34795)
0809 00:46:22.417620 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.417629 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.417694 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.417852 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.417852 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.417853 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:22.417899 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.417900 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.418735 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.418736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.420142 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.420145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.420916 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.420917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.420999 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.421003 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.421063 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.421121 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":126,"thread_start_us":145,"threads_started":1}
I20260809 00:46:22.421609 23483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0da710fa023549219e796b48477afd64" candidate_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" is_pre_election: true
I20260809 00:46:22.421813 23483 raft_consensus.cc:2454] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8d4c5d20e3b42b6bd66f8be283c3186 in term 0.
I20260809 00:46:22.421871 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0da710fa023549219e796b48477afd64" candidate_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5d3df756894acd8e8f5cd471c80468" is_pre_election: true
I20260809 00:46:22.422065 23880 raft_consensus.cc:2454] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8d4c5d20e3b42b6bd66f8be283c3186 in term 0.
I20260809 00:46:22.422048 23577 leader_election.cc:302] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbf4d36649a24bba9bbabe77dbe954a2, f8d4c5d20e3b42b6bd66f8be283c3186; no voters:
I20260809 00:46:22.422191 23981 raft_consensus.cc:2781] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:46:22.422256 23981 raft_consensus.cc:482] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:46:22.422282 23981 raft_consensus.cc:3037] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:22.423056 23981 raft_consensus.cc:504] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.423213 23981 leader_election.cc:288] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [CANDIDATE]: Term 1 election: Requested vote from peers 5e5d3df756894acd8e8f5cd471c80468 (127.30.104.195:34521), cbf4d36649a24bba9bbabe77dbe954a2 (127.30.104.193:37211)
I20260809 00:46:22.423398 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0da710fa023549219e796b48477afd64" candidate_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e5d3df756894acd8e8f5cd471c80468"
I20260809 00:46:22.423404 23483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0da710fa023549219e796b48477afd64" candidate_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf4d36649a24bba9bbabe77dbe954a2"
I20260809 00:46:22.423512 23483 raft_consensus.cc:3037] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:22.423512 23880 raft_consensus.cc:3037] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:22.424458 23880 raft_consensus.cc:2454] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8d4c5d20e3b42b6bd66f8be283c3186 in term 1.
I20260809 00:46:22.424463 23483 raft_consensus.cc:2454] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8d4c5d20e3b42b6bd66f8be283c3186 in term 1.
I20260809 00:46:22.424655 23577 leader_election.cc:302] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbf4d36649a24bba9bbabe77dbe954a2, f8d4c5d20e3b42b6bd66f8be283c3186; no voters:
I20260809 00:46:22.424800 23981 raft_consensus.cc:2781] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:22.425006 23981 raft_consensus.cc:686] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 1 LEADER]: Becoming Leader. State: Replica: f8d4c5d20e3b42b6bd66f8be283c3186, State: Running, Role: LEADER
I20260809 00:46:22.425138 23981 consensus_queue.cc:237] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } }
I20260809 00:46:22.425999 23271 catalog_manager.cc:5495] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8d4c5d20e3b42b6bd66f8be283c3186 (127.30.104.194). New cstate: current_term: 1 leader_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8d4c5d20e3b42b6bd66f8be283c3186" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 } health_report { overall_health: UNKNOWN } } }
W20260809 00:46:22.436144 23743 tablet.cc:2369] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:46:22.438740 23966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.436289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.30.104.194:39909)
0809 00:46:22.436333 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.436335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.436344 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:46:22.436433 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.436434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.436451 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.436532 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:46:22.436672 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.437149 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.437939 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.438140 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.438221 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.438396 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:46:22.438497 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:46:22.438605 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.438623 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.438645 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 00:46:22.440656 23483 raft_consensus.cc:1261] T 0da710fa023549219e796b48477afd64 P cbf4d36649a24bba9bbabe77dbe954a2 [term 1 FOLLOWER]: Refusing update from remote peer f8d4c5d20e3b42b6bd66f8be283c3186: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:22.440656 23880 raft_consensus.cc:1261] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 1 FOLLOWER]: Refusing update from remote peer f8d4c5d20e3b42b6bd66f8be283c3186: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:22.441049 23981 consensus_queue.cc:1035] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e5d3df756894acd8e8f5cd471c80468" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 34521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:22.441190 23983 consensus_queue.cc:1035] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf4d36649a24bba9bbabe77dbe954a2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 37211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:22.443485 23989 mvcc.cc:205] Tried to move back new op lower bound from 7316424222473748480 to 7316424222413979648. Current Snapshot: MvccSnapshot[applied={T|T < 7316424222473748480}]
I20260809 00:46:22.444008 23992 mvcc.cc:205] Tried to move back new op lower bound from 7316424222473748480 to 7316424222413979648. Current Snapshot: MvccSnapshot[applied={T|T < 7316424222473748480}]
I20260809 00:46:22.444800 23990 mvcc.cc:205] Tried to move back new op lower bound from 7316424222473748480 to 7316424222413979648. Current Snapshot: MvccSnapshot[applied={T|T < 7316424222473748480}]
I20260809 00:46:22.446137 23268 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:22.446251 23268 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:22.447113 23268 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8d44d66a0415467f93a9c924c622f66c: Sending DeleteTablet for 3 replicas of tablet 0da710fa023549219e796b48477afd64
I20260809 00:46:22.447561 23640 tablet_service.cc:1514] Processing DeleteTablet for tablet 0da710fa023549219e796b48477afd64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:49534
I20260809 00:46:22.447538 23850 tablet_service.cc:1514] Processing DeleteTablet for tablet 0da710fa023549219e796b48477afd64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:59570
I20260809 00:46:22.447799 24003 tablet_replica.cc:330] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: stopping tablet replica
I20260809 00:46:22.447798 23451 tablet_service.cc:1514] Processing DeleteTablet for tablet 0da710fa023549219e796b48477afd64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:22 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:55538
I20260809 00:46:22.447906 24003 raft_consensus.cc:2229] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:46:22.448001 24003 raft_consensus.cc:2258] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:22.448086 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 23367
I20260809 00:46:22.448153 24004 tablet_replica.cc:330] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: stopping tablet replica
I20260809 00:46:22.448235 24004 raft_consensus.cc:2229] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:22.448315 24004 raft_consensus.cc:2258] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:22.456496 24003 ts_tablet_manager.cc:1905] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:22.459615 24003 ts_tablet_manager.cc:1918] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:22.459777 24003 log.cc:1196] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/0da710fa023549219e796b48477afd64
I20260809 00:46:22.460160 24003 ts_tablet_manager.cc:1939] T 0da710fa023549219e796b48477afd64 P f8d4c5d20e3b42b6bd66f8be283c3186: Deleting consensus metadata
I20260809 00:46:22.460683 23249 catalog_manager.cc:4841] TS f8d4c5d20e3b42b6bd66f8be283c3186 (127.30.104.194:39909): tablet 0da710fa023549219e796b48477afd64 (table test-table [id=b7bde5aeb06e4542b3324f1bfb8db837]) successfully deleted
I20260809 00:46:22.465229 24004 ts_tablet_manager.cc:1905] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:22.466854 24004 ts_tablet_manager.cc:1918] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:22.466989 24004 log.cc:1196] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/0da710fa023549219e796b48477afd64
I20260809 00:46:22.467267 24004 ts_tablet_manager.cc:1939] T 0da710fa023549219e796b48477afd64 P 5e5d3df756894acd8e8f5cd471c80468: Deleting consensus metadata
I20260809 00:46:22.467803 23250 catalog_manager.cc:4841] TS 5e5d3df756894acd8e8f5cd471c80468 (127.30.104.195:34521): tablet 0da710fa023549219e796b48477afd64 (table test-table [id=b7bde5aeb06e4542b3324f1bfb8db837]) successfully deleted
W20260809 00:46:22.472219 23249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:37211 (error 108)
I20260809 00:46:22.472821 23963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.472679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:37211 (local address 127.0.0.1:55544)
0809 00:46:22.472738 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.472778 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:37211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 00:46:22.472886 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 23559
W20260809 00:46:22.478694 23249 catalog_manager.cc:4584] TS cbf4d36649a24bba9bbabe77dbe954a2 (127.30.104.193:37211): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0da710fa023549219e796b48477afd64: Network error: Client connection negotiation failed: client connection to 127.30.104.193:37211: connect: Connection refused (error 111)
I20260809 00:46:22.483227 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 23752
I20260809 00:46:22.491288 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 23225
2026-08-09T00:46:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2223 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T00:46:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:22Z Disabled control of system clock
I20260809 00:46:22.524777 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44189
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:44189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:46:22.626658 24011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:22.627022 24011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:22.627063 24011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:22.628923 24011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:22.628978 24011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:22.628993 24011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:22.629005 24011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:22.629017 24011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:22.631402 24011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:44189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:22.631790 24011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:22.632220 24011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:22.646298 24018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.646296 24022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.646298 24025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:22.647078 24011 server_base.cc:1034] running on GCE node
I20260809 00:46:22.647473 24011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:22.647879 24011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:22.649061 24011 hybrid_clock.cc:648] HybridClock initialized: now 1786236382649037 us; error 37 us; skew 500 ppm
I20260809 00:46:22.650672 24011 webserver.cc:460] Webserver started at http://127.30.104.254:46651/ using document root <none> and password file <none>
I20260809 00:46:22.650955 24011 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:22.651031 24011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:22.651197 24011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:22.652406 24011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "a1071bbcea79408e85189a2fc4779149"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "5dee22fcf1452569226bc6d494cf1000"
server_key_iv: "eb29dda61f958886b7698b168ad4ce9a"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.652812 24011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "a1071bbcea79408e85189a2fc4779149"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "5dee22fcf1452569226bc6d494cf1000"
server_key_iv: "eb29dda61f958886b7698b168ad4ce9a"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.654268 24011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:46:22.655181 24036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.655541 24011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:22.655659 24011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a1071bbcea79408e85189a2fc4779149"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "5dee22fcf1452569226bc6d494cf1000"
server_key_iv: "eb29dda61f958886b7698b168ad4ce9a"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.655756 24011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:22.679795 24011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:22.680176 24011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:22.680351 24011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:22.684962 24011 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:44189
I20260809 00:46:22.685019 24111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:44189 every 8 connection(s)
I20260809 00:46:22.685413 24011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:22.686451 24112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:22.688846 24112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149: Bootstrap starting.
I20260809 00:46:22.689461 24112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:22.689716 24112 log.cc:824] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:22.690397 24112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149: No bootstrap required, opened a new log
I20260809 00:46:22.692126 24112 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1071bbcea79408e85189a2fc4779149" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44189 } }
I20260809 00:46:22.692274 24112 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:22.692298 24112 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1071bbcea79408e85189a2fc4779149, State: Initialized, Role: FOLLOWER
I20260809 00:46:22.692443 24112 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1071bbcea79408e85189a2fc4779149" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44189 } }
I20260809 00:46:22.692536 24112 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:22.692571 24112 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:22.692601 24112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:22.693215 24112 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1071bbcea79408e85189a2fc4779149" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44189 } }
I20260809 00:46:22.693315 24112 leader_election.cc:302] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1071bbcea79408e85189a2fc4779149; no voters:
I20260809 00:46:22.693457 24112 leader_election.cc:288] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:22.693573 24117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:22.693765 24112 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:22.693820 24117 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [term 1 LEADER]: Becoming Leader. State: Replica: a1071bbcea79408e85189a2fc4779149, State: Running, Role: LEADER
I20260809 00:46:22.693939 24117 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1071bbcea79408e85189a2fc4779149" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44189 } }
I20260809 00:46:22.694509 24118 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1071bbcea79408e85189a2fc4779149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1071bbcea79408e85189a2fc4779149" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44189 } } }
I20260809 00:46:22.694648 24118 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:22.694617 24119 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1071bbcea79408e85189a2fc4779149. Latest consensus state: current_term: 1 leader_uuid: "a1071bbcea79408e85189a2fc4779149" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1071bbcea79408e85189a2fc4779149" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44189 } } }
I20260809 00:46:22.694758 24119 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:22.694883 24127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:22.695092 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 24011
I20260809 00:46:22.695247 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:22.695497 24127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:22.695505 31139 external_mini_cluster.cc:1428] Setting key 77c408d6db6f0f430841ecfebee53a2a
I20260809 00:46:22.697641 24127 catalog_manager.cc:1347] Generated new cluster ID: 34ed89a52b0a4c9fbdad14340ba05246
I20260809 00:46:22.697710 24127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:22.702553 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.696333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56674 (local address 127.30.104.254:44189)
0809 00:46:22.696539 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.696549 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.696583 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:46:22.696711 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.696718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.696805 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.697144 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:22.697273 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.698807 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.700165 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.700797 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.700920 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.701691 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:22.701695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:22.701695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:22.701928 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:22.701969 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:22.701983 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:22.702048 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.702161 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.702331 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260809 00:46:22.717034 24127 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:22.717760 24127 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:22.725957 24127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149: Generated new TSK 0
I20260809 00:46:22.726210 24127 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:22.728422 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44189
--builtin_ntp_servers=127.30.104.212:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:46:22.816012 24142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:22.816241 24142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:22.816267 24142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:22.817586 24142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:22.817636 24142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:22.817663 24142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:46:22.819144 24142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:22.819396 24142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:22.819730 24142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:22.825942 24153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.825958 24154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.825992 24159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:22.826300 24142 server_base.cc:1034] running on GCE node
I20260809 00:46:22.826503 24142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:22.826778 24142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:22.827962 24142 hybrid_clock.cc:648] HybridClock initialized: now 1786236382827918 us; error 57 us; skew 500 ppm
I20260809 00:46:22.829329 24142 webserver.cc:460] Webserver started at http://127.30.104.193:39927/ using document root <none> and password file <none>
I20260809 00:46:22.829594 24142 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:22.829646 24142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:22.829771 24142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:22.830897 24142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "db6d2b4643974203b1d243a4d1517ce2"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "36d629668cd15ad2486103d96ed9a246"
server_key_iv: "03a11e2f209666e74947164180937323"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.831269 24142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "db6d2b4643974203b1d243a4d1517ce2"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "36d629668cd15ad2486103d96ed9a246"
server_key_iv: "03a11e2f209666e74947164180937323"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.832755 24142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:22.833658 24166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.833901 24142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:22.834022 24142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "db6d2b4643974203b1d243a4d1517ce2"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "36d629668cd15ad2486103d96ed9a246"
server_key_iv: "03a11e2f209666e74947164180937323"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.834112 24142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:22.855780 24142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:22.856153 24142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:22.856325 24142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:22.856600 24142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:22.856957 24142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:22.857015 24142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.857082 24142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:22.857112 24142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.863261 24142 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:38737
I20260809 00:46:22.863320 24324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:38737 every 8 connection(s)
I20260809 00:46:22.863752 24142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:46:22.864877 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 24142
I20260809 00:46:22.865011 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:46:22.865201 31139 external_mini_cluster.cc:1428] Setting key 1cfc034ca6fb70f8624b29f344f3886c
I20260809 00:46:22.866238 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44189
--builtin_ntp_servers=127.30.104.212:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:46:22.870081 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.864618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:58627 (local address 127.30.104.254:44189)
0809 00:46:22.864688 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.864690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.864917 (+ 227us) server_negotiation.cc:407] Connection header received
0809 00:46:22.865668 (+ 751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.865670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.865686 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.865735 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:22.866601 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.867119 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.868544 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.868754 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.868820 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.869795 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:22.869796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:22.869796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:22.869801 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:22.869821 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:22.869825 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:22.869872 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.869982 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.870017 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:46:22.870393 24329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.864550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44189 (local address 127.30.104.193:58627)
0809 00:46:22.864785 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.864821 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.865564 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.865825 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.865829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.865845 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:22.866484 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.866491 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.867231 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.867232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.868437 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.868441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.868852 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.868854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.869029 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.869624 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:22.869624 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:22.869665 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:22.869696 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:22.869895 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:22.869902 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:22.869905 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.869977 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.870109 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260809 00:46:22.871006 24326 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44189
I20260809 00:46:22.871133 24326 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:22.871376 24326 heartbeater.cc:511] Master 127.30.104.254:44189 requested a full tablet report, sending...
I20260809 00:46:22.871974 24065 ts_manager.cc:194] Registered new tserver with Master: db6d2b4643974203b1d243a4d1517ce2 (127.30.104.193:38737)
I20260809 00:46:22.872641 24065 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.193:58627
W20260809 00:46:22.953256 24331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:22.953516 24331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:22.953562 24331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:22.954790 24331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:22.954866 24331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:22.954910 24331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:46:22.956488 24331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:22.956789 24331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:22.957127 24331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:22.962815 24342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.962847 24344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:22.962815 24341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:22.963138 24331 server_base.cc:1034] running on GCE node
I20260809 00:46:22.963521 24331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:22.963842 24331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:22.965008 24331 hybrid_clock.cc:648] HybridClock initialized: now 1786236382964984 us; error 50 us; skew 500 ppm
I20260809 00:46:22.966459 24331 webserver.cc:460] Webserver started at http://127.30.104.194:41909/ using document root <none> and password file <none>
I20260809 00:46:22.966754 24331 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:22.966807 24331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:22.966974 24331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:22.968253 24331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "a1b22f0daa2744e79bbf855eccf55b9a"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "aa696dd8431b0c6e74d7540604119c61"
server_key_iv: "7f3b4d9c634069b0f4ee74d2107c51c2"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.968647 24331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "a1b22f0daa2744e79bbf855eccf55b9a"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "aa696dd8431b0c6e74d7540604119c61"
server_key_iv: "7f3b4d9c634069b0f4ee74d2107c51c2"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.970148 24331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:46:22.971123 24352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.971354 24331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:22.971494 24331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "a1b22f0daa2744e79bbf855eccf55b9a"
format_stamp: "Formatted at 2026-08-09 00:46:22 on dist-test-slave-c12x"
server_key: "aa696dd8431b0c6e74d7540604119c61"
server_key_iv: "7f3b4d9c634069b0f4ee74d2107c51c2"
server_key_version: "encryptionkey@0"
I20260809 00:46:22.971616 24331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:22.985401 24331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:22.985792 24331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:22.985963 24331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:22.986224 24331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:22.986613 24331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:22.986670 24331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.986707 24331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:22.986755 24331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:22.993245 24331 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:39801
I20260809 00:46:22.993304 24508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:39801 every 8 connection(s)
I20260809 00:46:22.993674 24331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:46:22.999785 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.994586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:54471 (local address 127.30.104.254:44189)
0809 00:46:22.994715 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:22.994717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:22.994933 (+ 216us) server_negotiation.cc:407] Connection header received
0809 00:46:22.995569 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:22.995570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:22.995584 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:22.995637 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:22.996373 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.996879 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.997926 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.998150 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.998241 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.999185 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:22.999187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:22.999187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:22.999194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:22.999214 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:22.999225 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:22.999297 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:22.999647 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 00:46:22.999696 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 00:46:23.000052 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:22.994541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44189 (local address 127.30.104.194:54471)
0809 00:46:22.994790 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:22.994838 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:22.995372 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:22.995710 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:22.995714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:22.995726 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:22.996253 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.996257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.996990 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.996992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.997821 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:22.997824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:22.998255 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:22.998256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:22.998403 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:22.998991 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:22.998992 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:22.999023 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:22.999048 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:22.999298 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:22.999305 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:22.999309 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:46:22.999650 (+ 341us) client_negotiation.cc:240] Negotiation successful
0809 00:46:22.999771 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":111,"threads_started":1}
I20260809 00:46:23.000598 24509 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44189
I20260809 00:46:23.000689 24509 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:23.000897 24509 heartbeater.cc:511] Master 127.30.104.254:44189 requested a full tablet report, sending...
I20260809 00:46:23.001375 24065 ts_manager.cc:194] Registered new tserver with Master: a1b22f0daa2744e79bbf855eccf55b9a (127.30.104.194:39801)
I20260809 00:46:23.001796 24065 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.194:54471
I20260809 00:46:23.003944 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 24331
I20260809 00:46:23.004093 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:46:23.004321 31139 external_mini_cluster.cc:1428] Setting key 804347f2693126445efd7e2c2e3bb64b
I20260809 00:46:23.005622 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44189
--builtin_ntp_servers=127.30.104.212:36675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:46:23.096346 24514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:23.096583 24514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:23.096608 24514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:23.097844 24514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:23.097890 24514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:23.097919 24514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:46:23.099371 24514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:36675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:23.099699 24514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:23.100026 24514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:23.105540 24525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:23.105579 24514 server_base.cc:1034] running on GCE node
W20260809 00:46:23.105556 24523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:23.105556 24521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:23.105968 24514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:23.106242 24514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:23.107524 24514 hybrid_clock.cc:648] HybridClock initialized: now 1786236383107412 us; error 38 us; skew 500 ppm
I20260809 00:46:23.108891 24514 webserver.cc:460] Webserver started at http://127.30.104.195:41819/ using document root <none> and password file <none>
I20260809 00:46:23.109170 24514 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:23.109246 24514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:23.109406 24514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:23.110533 24514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "de9e773748d94b069d629fc8204a9295"
format_stamp: "Formatted at 2026-08-09 00:46:23 on dist-test-slave-c12x"
server_key: "c30b3cd6a4d14b82fe727e02cafcb0c0"
server_key_iv: "d563aeaff1603575936ea4b0a17df5eb"
server_key_version: "encryptionkey@0"
I20260809 00:46:23.110911 24514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "de9e773748d94b069d629fc8204a9295"
format_stamp: "Formatted at 2026-08-09 00:46:23 on dist-test-slave-c12x"
server_key: "c30b3cd6a4d14b82fe727e02cafcb0c0"
server_key_iv: "d563aeaff1603575936ea4b0a17df5eb"
server_key_version: "encryptionkey@0"
I20260809 00:46:23.112599 24514 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:23.113628 24534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:23.113992 24514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:23.114112 24514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "de9e773748d94b069d629fc8204a9295"
format_stamp: "Formatted at 2026-08-09 00:46:23 on dist-test-slave-c12x"
server_key: "c30b3cd6a4d14b82fe727e02cafcb0c0"
server_key_iv: "d563aeaff1603575936ea4b0a17df5eb"
server_key_version: "encryptionkey@0"
I20260809 00:46:23.114204 24514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:23.146648 24514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:23.147017 24514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:23.147183 24514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:23.147543 24514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:23.147926 24514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:23.147986 24514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:23.148025 24514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:23.148073 24514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:23.154033 24514 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:41693
I20260809 00:46:23.154076 24697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:41693 every 8 connection(s)
I20260809 00:46:23.154435 24514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:46:23.160172 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.155259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:45389 (local address 127.30.104.254:44189)
0809 00:46:23.155307 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:23.155308 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:23.155676 (+ 368us) server_negotiation.cc:407] Connection header received
0809 00:46:23.156239 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:23.156241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:23.156255 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:23.156312 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:23.157051 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.157565 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.158583 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.158788 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.158871 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.159849 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:23.159851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:23.159851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:23.159856 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:23.159872 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:23.159879 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:23.159945 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:23.160055 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:46:23.160088 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:46:23.160399 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.155239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44189 (local address 127.30.104.195:45389)
0809 00:46:23.155536 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:23.155586 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:23.156124 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:23.156385 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:23.156390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:23.156400 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:23.156925 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.156929 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.157667 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.157669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.158469 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.158471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.158881 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.158882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.159030 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.159692 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:23.159693 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:23.159722 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:23.159745 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:23.159962 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:23.159967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:23.159970 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:23.160056 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:46:23.160164 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260809 00:46:23.160928 24698 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44189
I20260809 00:46:23.161041 24698 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:23.161228 24698 heartbeater.cc:511] Master 127.30.104.254:44189 requested a full tablet report, sending...
I20260809 00:46:23.161638 24065 ts_manager.cc:194] Registered new tserver with Master: de9e773748d94b069d629fc8204a9295 (127.30.104.195:41693)
I20260809 00:46:23.162057 24065 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.195:45389
I20260809 00:46:23.163002 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 24514
I20260809 00:46:23.163095 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:46:23.163311 31139 external_mini_cluster.cc:1428] Setting key e92116fc8efb61a8d4585428e0d69aea
I20260809 00:46:23.164594 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:46:23.169066 24140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.166067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56686 (local address 127.30.104.254:44189)
0809 00:46:23.166115 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:23.166121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:23.166224 (+ 103us) server_negotiation.cc:407] Connection header received
0809 00:46:23.166281 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:23.166282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:23.166292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:23.166349 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:23.166466 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.166953 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.167790 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.167996 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.168071 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.168776 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:23.168778 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:23.168778 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:23.168783 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:23.168799 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:23.168803 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:23.168859 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:23.168969 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 00:46:23.168998 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:46:23.170849 24065 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:46:23.173156 24065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:46:23.181095 24711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.176341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:38737 (local address 127.0.0.1:56080)
0809 00:46:23.176895 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:23.176906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:23.176959 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:23.177357 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:23.177358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:23.177360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:23.177429 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.177432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.178671 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.178673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.179749 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.179752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.180695 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.180696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.180833 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.180852 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 00:46:23.180964 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 00:46:23.181024 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":131,"thread_start_us":92,"threads_started":1}
I20260809 00:46:23.181720 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.176537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56080 (local address 127.30.104.193:38737)
0809 00:46:23.177215 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:23.177218 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:23.177225 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:46:23.177255 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:23.177257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:23.177267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:23.177357 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:23.177529 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.178592 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.179844 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.180576 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.180664 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.180763 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:23.181556 (+ 793us) server_negotiation.cc:299] Negotiation successful
0809 00:46:23.181630 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":92,"threads_started":1}
I20260809 00:46:23.182096 24712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.176519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:41693 (local address 127.0.0.1:45462)
0809 00:46:23.177453 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:23.177463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:23.177511 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:23.177721 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:23.177722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:23.177724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:23.177760 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.177761 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.178629 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.178630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.180051 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.180055 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.181794 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.181796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.181908 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.181913 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:46:23.181978 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:46:23.182026 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":917,"mutex_wait_us":138,"thread_start_us":95,"threads_started":1}
I20260809 00:46:23.182402 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.176630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.30.104.195:41693)
0809 00:46:23.176836 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:23.176842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:23.177579 (+ 737us) server_negotiation.cc:407] Connection header received
0809 00:46:23.177621 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:23.177623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:23.177635 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:23.177712 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:23.177814 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.178512 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.180878 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.181691 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.181781 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.181886 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:23.182272 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 00:46:23.182320 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":126,"threads_started":1}
I20260809 00:46:23.182618 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.176281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:39801 (local address 127.0.0.1:53152)
0809 00:46:23.176552 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:23.176600 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:23.176676 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:23.177126 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:23.177129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:23.177131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:23.177191 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.177192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.178577 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.178578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.180051 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.180055 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.182286 (+ 2231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.182287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.182361 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.182365 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:46:23.182487 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:46:23.182542 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":103,"thread_start_us":75,"threads_started":1}
I20260809 00:46:23.182696 24232 tablet_service.cc:1467] Processing CreateTablet for tablet d2f3dff64c7e49c498d61744448f22af (DEFAULT_TABLE table=test-table [id=43107c7534224d9f91040209c0d5b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:23.183097 24232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f3dff64c7e49c498d61744448f22af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:23.183204 24604 tablet_service.cc:1467] Processing CreateTablet for tablet d2f3dff64c7e49c498d61744448f22af (DEFAULT_TABLE table=test-table [id=43107c7534224d9f91040209c0d5b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:23.183637 24604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f3dff64c7e49c498d61744448f22af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:23.186159 24716 tablet_bootstrap.cc:492] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2: Bootstrap starting.
I20260809 00:46:23.186564 24717 tablet_bootstrap.cc:492] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: Bootstrap starting.
I20260809 00:46:23.186868 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.176749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53152 (local address 127.30.104.194:39801)
0809 00:46:23.176965 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:23.176970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:23.176980 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:46:23.177022 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:23.177024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:23.177039 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:23.177113 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:23.177278 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.178482 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.180146 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.182097 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.182199 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.182291 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:23.186701 (+ 4410us) server_negotiation.cc:299] Negotiation successful
0809 00:46:23.186774 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":90,"threads_started":1}
I20260809 00:46:23.187031 24716 tablet_bootstrap.cc:654] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:23.187256 24717 tablet_bootstrap.cc:654] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:23.187407 24716 log.cc:824] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:23.187626 24717 log.cc:824] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:23.187683 24420 tablet_service.cc:1467] Processing CreateTablet for tablet d2f3dff64c7e49c498d61744448f22af (DEFAULT_TABLE table=test-table [id=43107c7534224d9f91040209c0d5b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:23.188088 24420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2f3dff64c7e49c498d61744448f22af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:23.188449 24716 tablet_bootstrap.cc:492] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2: No bootstrap required, opened a new log
I20260809 00:46:23.188534 24716 ts_tablet_manager.cc:1397] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:46:23.188690 24717 tablet_bootstrap.cc:492] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: No bootstrap required, opened a new log
I20260809 00:46:23.188773 24717 ts_tablet_manager.cc:1397] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 00:46:23.190724 24717 raft_consensus.cc:348] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.190896 24717 raft_consensus.cc:374] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:23.190943 24717 raft_consensus.cc:729] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9e773748d94b069d629fc8204a9295, State: Initialized, Role: FOLLOWER
I20260809 00:46:23.191067 24717 consensus_queue.cc:260] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.191356 24717 ts_tablet_manager.cc:1428] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:46:23.191567 24698 heartbeater.cc:503] Master 127.30.104.254:44189 was elected leader, sending a full tablet report...
I20260809 00:46:23.191455 24716 raft_consensus.cc:348] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.191615 24716 raft_consensus.cc:374] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:23.191653 24716 raft_consensus.cc:729] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db6d2b4643974203b1d243a4d1517ce2, State: Initialized, Role: FOLLOWER
I20260809 00:46:23.191771 24716 consensus_queue.cc:260] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.192041 24716 ts_tablet_manager.cc:1428] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:46:23.192160 24326 heartbeater.cc:503] Master 127.30.104.254:44189 was elected leader, sending a full tablet report...
I20260809 00:46:23.192678 24723 tablet_bootstrap.cc:492] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Bootstrap starting.
I20260809 00:46:23.193361 24723 tablet_bootstrap.cc:654] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:23.193614 24723 log.cc:824] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:23.194360 24723 tablet_bootstrap.cc:492] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: No bootstrap required, opened a new log
I20260809 00:46:23.194478 24723 ts_tablet_manager.cc:1397] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:46:23.196442 24723 raft_consensus.cc:348] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.196578 24723 raft_consensus.cc:374] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:23.196599 24723 raft_consensus.cc:729] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1b22f0daa2744e79bbf855eccf55b9a, State: Initialized, Role: FOLLOWER
I20260809 00:46:23.196718 24723 consensus_queue.cc:260] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.196949 24723 ts_tablet_manager.cc:1428] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 00:46:23.197070 24509 heartbeater.cc:503] Master 127.30.104.254:44189 was elected leader, sending a full tablet report...
W20260809 00:46:23.244277 24510 tablet.cc:2369] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:46:23.300735 24722 raft_consensus.cc:482] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:23.300915 24722 raft_consensus.cc:504] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.301433 24722 leader_election.cc:288] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1b22f0daa2744e79bbf855eccf55b9a (127.30.104.194:39801), db6d2b4643974203b1d243a4d1517ce2 (127.30.104.193:38737)
I20260809 00:46:23.304997 24715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.301785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:56305 (local address 127.30.104.194:39801)
0809 00:46:23.301865 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:23.301868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:23.302210 (+ 342us) server_negotiation.cc:407] Connection header received
0809 00:46:23.302253 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:23.302254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:23.302266 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:23.302316 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:23.302525 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.302974 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.304188 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.304592 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.304653 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.304659 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:23.304868 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 00:46:23.304918 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:46:23.305014 24727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.301679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:39801 (local address 127.30.104.195:56305)
0809 00:46:23.302073 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:23.302085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:23.302165 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:23.302346 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:23.302347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:23.302350 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:23.302437 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.302439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.303091 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.303091 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.304077 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.304080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.304692 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.304693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.304775 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.304780 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:46:23.304850 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:46:23.304886 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":181,"thread_start_us":115,"threads_started":1}
I20260809 00:46:23.305123 24713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.301826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:44485 (local address 127.30.104.193:38737)
0809 00:46:23.301909 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:23.301911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:23.301923 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:46:23.301956 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:23.301957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:23.301972 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:23.302034 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:23.302324 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.302948 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.304255 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.304684 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.304771 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.304778 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:23.304994 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 00:46:23.305049 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:46:23.305370 24701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.301679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:38737 (local address 127.30.104.195:44485)
0809 00:46:23.301727 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:23.301740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:23.301787 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:23.302061 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:23.302063 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:23.302065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:23.302162 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.302164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.303049 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.303050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.304101 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:23.304104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.304789 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.304790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:23.304882 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.304885 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:23.304950 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:46:23.304991 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":46}
I20260809 00:46:23.305543 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f3dff64c7e49c498d61744448f22af" candidate_uuid: "de9e773748d94b069d629fc8204a9295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" is_pre_election: true
I20260809 00:46:23.305727 24446 raft_consensus.cc:2454] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de9e773748d94b069d629fc8204a9295 in term 0.
I20260809 00:46:23.305778 24261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f3dff64c7e49c498d61744448f22af" candidate_uuid: "de9e773748d94b069d629fc8204a9295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db6d2b4643974203b1d243a4d1517ce2" is_pre_election: true
I20260809 00:46:23.305948 24261 raft_consensus.cc:2454] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de9e773748d94b069d629fc8204a9295 in term 0.
I20260809 00:46:23.305970 24537 leader_election.cc:302] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1b22f0daa2744e79bbf855eccf55b9a, de9e773748d94b069d629fc8204a9295; no voters:
I20260809 00:46:23.306103 24722 raft_consensus.cc:2781] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:46:23.306165 24722 raft_consensus.cc:482] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:46:23.306197 24722 raft_consensus.cc:3037] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:23.306958 24722 raft_consensus.cc:504] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.307129 24722 leader_election.cc:288] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [CANDIDATE]: Term 1 election: Requested vote from peers a1b22f0daa2744e79bbf855eccf55b9a (127.30.104.194:39801), db6d2b4643974203b1d243a4d1517ce2 (127.30.104.193:38737)
I20260809 00:46:23.307294 24261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f3dff64c7e49c498d61744448f22af" candidate_uuid: "de9e773748d94b069d629fc8204a9295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db6d2b4643974203b1d243a4d1517ce2"
I20260809 00:46:23.307324 24446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2f3dff64c7e49c498d61744448f22af" candidate_uuid: "de9e773748d94b069d629fc8204a9295" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1b22f0daa2744e79bbf855eccf55b9a"
I20260809 00:46:23.307376 24261 raft_consensus.cc:3037] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:23.307420 24446 raft_consensus.cc:3037] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:23.308187 24446 raft_consensus.cc:2454] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de9e773748d94b069d629fc8204a9295 in term 1.
I20260809 00:46:23.308391 24261 raft_consensus.cc:2454] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de9e773748d94b069d629fc8204a9295 in term 1.
I20260809 00:46:23.308424 24537 leader_election.cc:302] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a1b22f0daa2744e79bbf855eccf55b9a, de9e773748d94b069d629fc8204a9295; no voters:
I20260809 00:46:23.308526 24722 raft_consensus.cc:2781] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:23.308686 24722 raft_consensus.cc:686] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 1 LEADER]: Becoming Leader. State: Replica: de9e773748d94b069d629fc8204a9295, State: Running, Role: LEADER
I20260809 00:46:23.308804 24722 consensus_queue.cc:237] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } }
I20260809 00:46:23.309741 24065 catalog_manager.cc:5495] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 reported cstate change: term changed from 0 to 1, leader changed from <none> to de9e773748d94b069d629fc8204a9295 (127.30.104.195). New cstate: current_term: 1 leader_uuid: "de9e773748d94b069d629fc8204a9295" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de9e773748d94b069d629fc8204a9295" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 41693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 } health_report { overall_health: UNKNOWN } } }
I20260809 00:46:23.326985 24714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.324413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45476 (local address 127.30.104.195:41693)
0809 00:46:23.324487 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:23.324489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:23.324500 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:46:23.324588 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:23.324589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:23.324607 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:23.324688 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:46:23.324850 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.325343 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.326129 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:23.326330 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:23.326411 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:23.326610 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:46:23.326717 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:46:23.326818 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:23.326838 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 00:46:23.326856 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:46:23.328980 24446 raft_consensus.cc:1261] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 1 FOLLOWER]: Refusing update from remote peer de9e773748d94b069d629fc8204a9295: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:23.329017 24261 raft_consensus.cc:1261] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 1 FOLLOWER]: Refusing update from remote peer de9e773748d94b069d629fc8204a9295: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:23.329397 24730 consensus_queue.cc:1035] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1b22f0daa2744e79bbf855eccf55b9a" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 39801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:23.329573 24722 consensus_queue.cc:1035] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [LEADER]: Connected to new peer: Peer: permanent_uuid: "db6d2b4643974203b1d243a4d1517ce2" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:23.332590 24740 mvcc.cc:205] Tried to move back new op lower bound from 7316424226112528384 to 7316424226033606656. Current Snapshot: MvccSnapshot[applied={T|T < 7316424226112528384}]
I20260809 00:46:23.332578 24738 mvcc.cc:205] Tried to move back new op lower bound from 7316424226112528384 to 7316424226033606656. Current Snapshot: MvccSnapshot[applied={T|T < 7316424226112528384}]
I20260809 00:46:23.334915 24063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:23.335021 24063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:23.335875 24063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a1071bbcea79408e85189a2fc4779149: Sending DeleteTablet for 3 replicas of tablet d2f3dff64c7e49c498d61744448f22af
I20260809 00:46:23.336267 24232 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f3dff64c7e49c498d61744448f22af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:23 UTC) from {username='slave'} at 127.0.0.1:56080
I20260809 00:46:23.336272 24420 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f3dff64c7e49c498d61744448f22af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:23 UTC) from {username='slave'} at 127.0.0.1:53152
I20260809 00:46:23.336484 24604 tablet_service.cc:1514] Processing DeleteTablet for tablet d2f3dff64c7e49c498d61744448f22af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:23 UTC) from {username='slave'} at 127.0.0.1:45462
I20260809 00:46:23.336577 24751 tablet_replica.cc:330] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2: stopping tablet replica
I20260809 00:46:23.336688 24751 raft_consensus.cc:2229] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:23.336702 24753 tablet_replica.cc:330] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: stopping tablet replica
I20260809 00:46:23.336783 24753 raft_consensus.cc:2229] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:23.336798 24751 raft_consensus.cc:2258] T d2f3dff64c7e49c498d61744448f22af P db6d2b4643974203b1d243a4d1517ce2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:23.336824 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 24142
I20260809 00:46:23.336863 24753 raft_consensus.cc:2258] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:23.337558 24754 tablet_replica.cc:330] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: stopping tablet replica
I20260809 00:46:23.337647 24754 raft_consensus.cc:2229] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:46:23.337741 24754 raft_consensus.cc:2258] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:23.339921 24754 ts_tablet_manager.cc:1905] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:23.341741 24754 ts_tablet_manager.cc:1918] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:23.341818 24754 log.cc:1196] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/d2f3dff64c7e49c498d61744448f22af
I20260809 00:46:23.342079 24754 ts_tablet_manager.cc:1939] T d2f3dff64c7e49c498d61744448f22af P de9e773748d94b069d629fc8204a9295: Deleting consensus metadata
I20260809 00:46:23.342546 24753 ts_tablet_manager.cc:1905] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:23.342547 24037 catalog_manager.cc:4841] TS de9e773748d94b069d629fc8204a9295 (127.30.104.195:41693): tablet d2f3dff64c7e49c498d61744448f22af (table test-table [id=43107c7534224d9f91040209c0d5b9b1]) successfully deleted
I20260809 00:46:23.344938 24753 ts_tablet_manager.cc:1918] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:23.345067 24753 log.cc:1196] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/d2f3dff64c7e49c498d61744448f22af
I20260809 00:46:23.345369 24753 ts_tablet_manager.cc:1939] T d2f3dff64c7e49c498d61744448f22af P a1b22f0daa2744e79bbf855eccf55b9a: Deleting consensus metadata
I20260809 00:46:23.345943 24037 catalog_manager.cc:4841] TS a1b22f0daa2744e79bbf855eccf55b9a (127.30.104.194:39801): tablet d2f3dff64c7e49c498d61744448f22af (table test-table [id=43107c7534224d9f91040209c0d5b9b1]) successfully deleted
W20260809 00:46:23.354158 24039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:38737 (error 108)
I20260809 00:46:23.354912 24710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:23.354639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:38737 (local address 127.0.0.1:56088)
0809 00:46:23.354799 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:23.354864 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:38737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260809 00:46:23.355034 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 24331
W20260809 00:46:23.360913 24039 catalog_manager.cc:4584] TS db6d2b4643974203b1d243a4d1517ce2 (127.30.104.193:38737): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d2f3dff64c7e49c498d61744448f22af: Network error: Client connection negotiation failed: client connection to 127.30.104.193:38737: connect: Connection refused (error 111)
I20260809 00:46:23.367451 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 24514
I20260809 00:46:23.381812 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 24011
2026-08-09T00:46:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (893 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:23 dist-test-slave-c12x krb5kdc[24761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:23 dist-test-slave-c12x krb5kdc[24761](info): set up 2 sockets
Aug 09 00:46:23 dist-test-slave-c12x krb5kdc[24761](info): commencing operation
krb5kdc: starting...
W20260809 00:46:24.622231 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:24 dist-test-slave-c12x krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:24.767361 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:42391
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:37469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:42391
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:24.855120 24779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:24.855343 24779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:24.855367 24779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:24.856678 24779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:24.856729 24779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:24.856743 24779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:24.856756 24779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:24.856769 24779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:24.858280 24779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:42391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:42391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:24.858496 24779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:24.858700 24779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:24.863806 24788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:24.863806 24789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:24.864096 24791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:24.864372 24779 server_base.cc:1034] running on GCE node
I20260809 00:46:24.864636 24779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:24.864969 24779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:24.866135 24779 hybrid_clock.cc:648] HybridClock initialized: now 1786236384866110 us; error 47 us; skew 500 ppm
Aug 09 00:46:24 dist-test-slave-c12x krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236384, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:24.868906 24779 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:24.869436 24779 webserver.cc:460] Webserver started at http://127.30.104.254:39925/ using document root <none> and password file <none>
I20260809 00:46:24.869697 24779 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:24.869771 24779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:24.869926 24779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:24.871029 24779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "f16dffa105864f17be33317279428d3d"
format_stamp: "Formatted at 2026-08-09 00:46:24 on dist-test-slave-c12x"
server_key: "83133c82615c183137e8d54ed5587280"
server_key_iv: "b3efbe444bc7fb250757f99288d9591f"
server_key_version: "encryptionkey@0"
I20260809 00:46:24.871413 24779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "f16dffa105864f17be33317279428d3d"
format_stamp: "Formatted at 2026-08-09 00:46:24 on dist-test-slave-c12x"
server_key: "83133c82615c183137e8d54ed5587280"
server_key_iv: "b3efbe444bc7fb250757f99288d9591f"
server_key_version: "encryptionkey@0"
I20260809 00:46:24.872838 24779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:24.873756 24801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:24.873971 24779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:46:24.874092 24779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "f16dffa105864f17be33317279428d3d"
format_stamp: "Formatted at 2026-08-09 00:46:24 on dist-test-slave-c12x"
server_key: "83133c82615c183137e8d54ed5587280"
server_key_iv: "b3efbe444bc7fb250757f99288d9591f"
server_key_version: "encryptionkey@0"
I20260809 00:46:24.874173 24779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:24.891980 24779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:24.894655 24779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:24.894830 24779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:24.898924 24779 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:42391
I20260809 00:46:24.898988 24877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:42391 every 8 connection(s)
I20260809 00:46:24.899385 24779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:24.900465 24878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:24.902842 24878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d: Bootstrap starting.
I20260809 00:46:24.903549 24878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:24.903863 24878 log.cc:824] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:24.904037 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 24779
I20260809 00:46:24.904177 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:24.904373 31139 external_mini_cluster.cc:1428] Setting key a93916a84b76321b1dc2ff64ff7258aa
I20260809 00:46:24.904722 24878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d: No bootstrap required, opened a new log
I20260809 00:46:24.906556 24878 raft_consensus.cc:348] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16dffa105864f17be33317279428d3d" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 42391 } }
I20260809 00:46:24.906725 24878 raft_consensus.cc:374] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:24.906773 24878 raft_consensus.cc:729] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f16dffa105864f17be33317279428d3d, State: Initialized, Role: FOLLOWER
I20260809 00:46:24.906926 24878 consensus_queue.cc:260] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16dffa105864f17be33317279428d3d" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 42391 } }
I20260809 00:46:24.907021 24878 raft_consensus.cc:388] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:24.907068 24878 raft_consensus.cc:482] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:24.907121 24878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:24.908180 24878 raft_consensus.cc:504] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16dffa105864f17be33317279428d3d" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 42391 } }
I20260809 00:46:24.908304 24878 leader_election.cc:302] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f16dffa105864f17be33317279428d3d; no voters:
I20260809 00:46:24.908528 24878 leader_election.cc:288] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:24.908552 24886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:24.908671 24886 raft_consensus.cc:686] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [term 1 LEADER]: Becoming Leader. State: Replica: f16dffa105864f17be33317279428d3d, State: Running, Role: LEADER
I20260809 00:46:24.908759 24886 consensus_queue.cc:237] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16dffa105864f17be33317279428d3d" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 42391 } }
I20260809 00:46:24.908815 24878 sys_catalog.cc:564] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:24.909165 24887 sys_catalog.cc:455] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f16dffa105864f17be33317279428d3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16dffa105864f17be33317279428d3d" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 42391 } } }
I20260809 00:46:24.909179 24886 sys_catalog.cc:455] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f16dffa105864f17be33317279428d3d. Latest consensus state: current_term: 1 leader_uuid: "f16dffa105864f17be33317279428d3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f16dffa105864f17be33317279428d3d" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 42391 } } }
I20260809 00:46:24.909363 24887 sys_catalog.cc:458] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:24.909382 24886 sys_catalog.cc:458] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:24.909724 24895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:24.910274 24895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:24.912370 24895 catalog_manager.cc:1347] Generated new cluster ID: 32c52bf6e6184668a992d3734d6bfcfc
I20260809 00:46:24.912432 24895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:46:24 dist-test-slave-c12x krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236384, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:24.917410 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:24.904982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41680 (local address 127.30.104.254:42391)
0809 00:46:24.905201 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:24.905216 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:24.905242 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:46:24.906566 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:24.906575 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:24.906631 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:24.906796 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:24.907593 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:24.908761 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:24.909892 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:24.910578 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:24.910723 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:24.913471 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:24.913477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:24.913478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:24.913496 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:24.915594 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:24.916033 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:24.916037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:24.916037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:24.916148 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:24.916394 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:24.916395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:24.916395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:24.916680 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:24.916863 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:24.916990 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 00:46:24.917163 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260809 00:46:24.933825 24895 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:24.934504 24895 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:24.939567 24895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d: Generated new TSK 0
I20260809 00:46:24.939906 24895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:46:24.974560 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:42391
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:37469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:25.065192 24911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:25.065411 24911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:25.065438 24911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:25.066612 24911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:25.066658 24911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:25.066685 24911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:46:25.068245 24911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:25.068517 24911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:25.068756 24911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:25.074280 24920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:25.074271 24926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:25.074271 24919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:25.074493 24911 server_base.cc:1034] running on GCE node
I20260809 00:46:25.074770 24911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:25.075047 24911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:25.076193 24911 hybrid_clock.cc:648] HybridClock initialized: now 1786236385076176 us; error 37 us; skew 500 ppm
Aug 09 00:46:25 dist-test-slave-c12x krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:25.079110 24911 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:46:25.079716 24911 webserver.cc:460] Webserver started at http://127.30.104.193:34993/ using document root <none> and password file <none>
I20260809 00:46:25.079960 24911 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:25.080013 24911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:25.080120 24911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:25.081359 24911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "aaef25430a9545de88642ba6b610ee91"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "ecd27211ef49620747d30bcc76cc9213"
server_key_iv: "ca8dfbe2f535915af33eacc03ff5483a"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.081722 24911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "aaef25430a9545de88642ba6b610ee91"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "ecd27211ef49620747d30bcc76cc9213"
server_key_iv: "ca8dfbe2f535915af33eacc03ff5483a"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.083238 24911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:25.084229 24942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.084467 24911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:25.084578 24911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "aaef25430a9545de88642ba6b610ee91"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "ecd27211ef49620747d30bcc76cc9213"
server_key_iv: "ca8dfbe2f535915af33eacc03ff5483a"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.084672 24911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:25.093319 24911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:25.095835 24911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:25.095997 24911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:25.096284 24911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:25.096633 24911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:25.096671 24911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.096694 24911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:25.096706 24911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.102913 24911 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:43753
I20260809 00:46:25.103011 25099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:43753 every 8 connection(s)
I20260809 00:46:25.103363 24911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
Aug 09 00:46:25 dist-test-slave-c12x krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:25.111146 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 24911
I20260809 00:46:25.111307 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:46:25.111636 31139 external_mini_cluster.cc:1428] Setting key c6f8583bc563482d6df921e65ce6b839
I20260809 00:46:25.114408 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.104227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:37441 (local address 127.30.104.254:42391)
0809 00:46:25.104295 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.104297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.104567 (+ 270us) server_negotiation.cc:407] Connection header received
0809 00:46:25.105045 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.105046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.105058 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.105124 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:25.106146 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.106714 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.107898 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.108118 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.108183 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.110594 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:25.110596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:25.110597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:25.110608 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:25.112771 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:25.113332 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:25.113333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:25.113333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:25.113392 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:25.113690 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:25.113691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:25.113691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:25.113806 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:25.113844 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.114249 (+ 405us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.114338 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 00:46:25.114763 25102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.104186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:42391 (local address 127.30.104.193:37441)
0809 00:46:25.104380 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.104425 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:25.104932 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:25.105205 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:25.105208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:25.105835 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:25.106038 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.106043 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.106835 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.106836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.107814 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.107817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.108201 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.108202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.108300 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.108873 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:25.108882 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:25.110479 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:25.112866 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:25.112877 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:25.112880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:25.113251 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:25.113461 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:25.113462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:25.113462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:25.113625 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:25.113883 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:25.113892 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:25.114097 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 00:46:25.114222 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 00:46:25.114438 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
I20260809 00:46:25.115463 25100 heartbeater.cc:347] Connected to a master server at 127.30.104.254:42391
I20260809 00:46:25.115942 25100 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:25.116220 25100 heartbeater.cc:511] Master 127.30.104.254:42391 requested a full tablet report, sending...
I20260809 00:46:25.116896 24826 ts_manager.cc:194] Registered new tserver with Master: aaef25430a9545de88642ba6b610ee91 (127.30.104.193:43753)
I20260809 00:46:25.117548 24826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:37441
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:46:25.140165 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:42391
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:37469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:25.225771 25111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:25.225989 25111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:25.226013 25111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:25.227164 25111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:25.227206 25111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:25.227259 25111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:46:25.228756 25111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:25.229012 25111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:25.229279 25111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:25.235208 25122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:25.235195 25123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:25.235198 25125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:25.235479 25111 server_base.cc:1034] running on GCE node
I20260809 00:46:25.235685 25111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:25.235967 25111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:25.237128 25111 hybrid_clock.cc:648] HybridClock initialized: now 1786236385237106 us; error 43 us; skew 500 ppm
Aug 09 00:46:25 dist-test-slave-c12x krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:25.239916 25111 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:46:25.240403 25111 webserver.cc:460] Webserver started at http://127.30.104.194:43389/ using document root <none> and password file <none>
I20260809 00:46:25.240615 25111 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:25.240656 25111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:25.240748 25111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:25.241796 25111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "cf091ebd96654b119c34e13cf4856b47"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "54cd9556a4dfa0415948556adbb568ae"
server_key_iv: "43959e49654e30f03a417c7bcb498fa8"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.242115 25111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "cf091ebd96654b119c34e13cf4856b47"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "54cd9556a4dfa0415948556adbb568ae"
server_key_iv: "43959e49654e30f03a417c7bcb498fa8"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.243417 25111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:46:25.244280 25132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.244483 25111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:25.244597 25111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "cf091ebd96654b119c34e13cf4856b47"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "54cd9556a4dfa0415948556adbb568ae"
server_key_iv: "43959e49654e30f03a417c7bcb498fa8"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.244679 25111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:25.266047 25111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:25.268270 25111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:25.268496 25111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:25.268772 25111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:25.269120 25111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:25.269181 25111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.269218 25111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:25.269268 25111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.275949 25111 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:36219
I20260809 00:46:25.276021 25302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:36219 every 8 connection(s)
I20260809 00:46:25.276422 25111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
Aug 09 00:46:25 dist-test-slave-c12x krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:25.286267 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.277256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:38899 (local address 127.30.104.254:42391)
0809 00:46:25.277321 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.277323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.277543 (+ 220us) server_negotiation.cc:407] Connection header received
0809 00:46:25.278189 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.278191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.278201 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.278265 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:25.279219 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.279829 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.280875 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.281097 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.281164 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.283499 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:25.283501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:25.283515 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:25.283523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:25.284906 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:25.285314 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:25.285315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:25.285315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:25.285368 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:25.285588 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:25.285588 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:25.285589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:25.285699 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:25.285752 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.286120 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.286197 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:46:25.286533 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 25111
I20260809 00:46:25.286648 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.277234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:42391 (local address 127.30.104.194:38899)
0809 00:46:25.277395 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.277439 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:25.278070 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:25.278325 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:25.278328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:25.278921 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:25.279127 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.279132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.279922 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.279923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.280777 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.280780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.281182 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.281183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.281292 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.281871 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:25.281879 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:25.283328 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:25.284998 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:25.285004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:25.285006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:25.285243 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:25.285429 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:25.285430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:25.285430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:25.285529 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:25.285755 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:25.285759 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:25.285946 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 00:46:25.286110 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 00:46:25.286308 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260809 00:46:25.286674 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:46:25.286871 31139 external_mini_cluster.cc:1428] Setting key 7ee7bf7c8ef58a6b73627f40f19f4284
I20260809 00:46:25.287302 25303 heartbeater.cc:347] Connected to a master server at 127.30.104.254:42391
I20260809 00:46:25.287415 25303 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:25.287647 25303 heartbeater.cc:511] Master 127.30.104.254:42391 requested a full tablet report, sending...
I20260809 00:46:25.288157 24826 ts_manager.cc:194] Registered new tserver with Master: cf091ebd96654b119c34e13cf4856b47 (127.30.104.194:36219)
I20260809 00:46:25.288717 24826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:38899
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:46:25.315424 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:42391
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:37469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:25.402509 25310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:25.402724 25310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:25.402747 25310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:25.403846 25310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:25.403882 25310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:25.403908 25310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:46:25.405313 25310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:37469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:25.405509 25310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:25.405704 25310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:25.410982 25315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:25.411058 25316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:25.410986 25319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:25.411746 25310 server_base.cc:1034] running on GCE node
I20260809 00:46:25.411978 25310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:25.412248 25310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:25.413455 25310 hybrid_clock.cc:648] HybridClock initialized: now 1786236385413428 us; error 44 us; skew 500 ppm
Aug 09 00:46:25 dist-test-slave-c12x krb5kdc[24761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:25.420193 25310 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:46:25.420789 25310 webserver.cc:460] Webserver started at http://127.30.104.195:40349/ using document root <none> and password file <none>
I20260809 00:46:25.421092 25310 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:25.421167 25310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:25.421324 25310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:25.422425 25310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "052b179cbfa8cff33f0a6319435874c1"
server_key_iv: "21a4bb3e86033fa9a29306083886009d"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.422796 25310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "052b179cbfa8cff33f0a6319435874c1"
server_key_iv: "21a4bb3e86033fa9a29306083886009d"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.424293 25310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:46:25.425077 25327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.425386 25310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:25.425499 25310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "052b179cbfa8cff33f0a6319435874c1"
server_key_iv: "21a4bb3e86033fa9a29306083886009d"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.425575 25310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:25.438972 25310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:25.441435 25310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:25.441613 25310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:25.441869 25310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:25.442200 25310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:25.442256 25310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.442291 25310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:25.442339 25310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.448436 25310 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:43305
I20260809 00:46:25.448515 25500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:43305 every 8 connection(s)
I20260809 00:46:25.448897 25310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:46:25.452560 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 25310
I20260809 00:46:25.452661 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:46:25.452859 31139 external_mini_cluster.cc:1428] Setting key 2f013db69582e5d91520493369725eeb
Aug 09 00:46:25 dist-test-slave-c12x krb5kdc[24761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236385, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:25.459249 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.449774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:51185 (local address 127.30.104.254:42391)
0809 00:46:25.449860 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.449863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.450135 (+ 272us) server_negotiation.cc:407] Connection header received
0809 00:46:25.450715 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.450716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.450726 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.450781 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:25.451806 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.452336 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.453901 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.454097 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.454153 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.456573 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:25.456574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:25.456575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:25.456582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:25.457884 (+ 1302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:25.458303 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:25.458304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:25.458305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:25.458361 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:25.458592 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:25.458592 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:25.458593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:25.458708 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:25.458762 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.459106 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.459181 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 00:46:25.459609 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.449724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:42391 (local address 127.30.104.195:51185)
0809 00:46:25.449991 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.450042 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:25.450607 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:25.450841 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:25.450844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:25.451487 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:25.451704 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.451709 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.452490 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.452491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.453815 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.453818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.454177 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.454178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.454287 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.454913 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:25.454925 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:25.456449 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:25.457979 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:25.457986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:25.457988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:25.458226 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:25.458428 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:25.458429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:25.458429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:25.458534 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:25.458795 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:25.458802 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:25.458964 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 00:46:25.459080 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 00:46:25.459224 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":120,"threads_started":1}
I20260809 00:46:25.460291 25501 heartbeater.cc:347] Connected to a master server at 127.30.104.254:42391
I20260809 00:46:25.460403 25501 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:25.460564 25501 heartbeater.cc:511] Master 127.30.104.254:42391 requested a full tablet report, sending...
I20260809 00:46:25.461001 24826 ts_manager.cc:194] Registered new tserver with Master: 5b0a8aa92ef04ecbaa6bc40fbe0ec26c (127.30.104.195:43305)
I20260809 00:46:25.461440 24826 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:51185
I20260809 00:46:25.464933 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:46:25.473420 24883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.466436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41696 (local address 127.30.104.254:42391)
0809 00:46:25.466493 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.466494 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.466663 (+ 169us) server_negotiation.cc:407] Connection header received
0809 00:46:25.466748 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.466749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.466760 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.466811 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:25.467390 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.467951 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.468755 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.468980 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.469062 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.470541 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:25.470542 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:25.470543 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:25.470551 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:25.472050 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:25.472475 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:25.472476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:25.472476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:25.472538 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:25.472765 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:25.472766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:25.472767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:25.472925 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:25.473049 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.473211 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.473282 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 00:46:25.475562 24826 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:46:25.477357 24826 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:46:25.485010 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.480564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.30.104.195:43305)
0809 00:46:25.480885 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.480891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.480959 (+ 68us) server_negotiation.cc:407] Connection header received
0809 00:46:25.481133 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.481136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.481154 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.481238 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:25.481510 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.482216 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.483950 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.484640 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.484709 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.484754 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.484870 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.484919 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":107,"threads_started":1}
I20260809 00:46:25.485020 25523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.480433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:43305 (local address 127.0.0.1:56370)
0809 00:46:25.480876 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.480890 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:25.480968 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:25.481339 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:25.481340 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:25.481342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:25.481386 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.481388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.482325 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.482327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.483855 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.483858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.484716 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.484717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.484763 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.484766 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:25.484845 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:46:25.484923 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"mutex_wait_us":80,"thread_start_us":75,"threads_started":1}
I20260809 00:46:25.485019 25519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.480190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:36219 (local address 127.0.0.1:57324)
0809 00:46:25.480527 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.480573 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:25.480647 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:25.481431 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:25.481431 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:25.481433 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:25.481469 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.481471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.482685 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.482686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.483711 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.483714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.484651 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.484652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.484749 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.484766 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 00:46:25.484845 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:46:25.484919 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":38,"thread_start_us":63,"threads_started":1}
I20260809 00:46:25.485898 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.480190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:43753 (local address 127.0.0.1:33596)
0809 00:46:25.480564 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.480573 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:25.480632 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:25.481042 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:25.481045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:25.481048 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:25.481126 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.481129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.482478 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.482478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.483855 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.483858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.485658 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.485659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.485738 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.485750 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 00:46:25.485794 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 00:46:25.485823 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":93,"thread_start_us":105,"threads_started":1}
I20260809 00:46:25.486228 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.480321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57324 (local address 127.30.104.194:36219)
0809 00:46:25.481211 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.481216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.481226 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:46:25.481265 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.481266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.481281 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.481352 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:25.481561 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.482586 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.483804 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.484559 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.484646 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.484712 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.485932 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.486148 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":845,"thread_start_us":99,"threads_started":1}
I20260809 00:46:25.486301 25402 tablet_service.cc:1467] Processing CreateTablet for tablet bd884919aec342728beb9db3cf149863 (DEFAULT_TABLE table=test-table [id=c4001f325b894c2e939cff3558b4afca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:25.486713 25402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd884919aec342728beb9db3cf149863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:25.486856 25206 tablet_service.cc:1467] Processing CreateTablet for tablet bd884919aec342728beb9db3cf149863 (DEFAULT_TABLE table=test-table [id=c4001f325b894c2e939cff3558b4afca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:25.487196 25206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd884919aec342728beb9db3cf149863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:25.490460 25525 tablet_bootstrap.cc:492] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Bootstrap starting.
I20260809 00:46:25.490525 25526 tablet_bootstrap.cc:492] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Bootstrap starting.
I20260809 00:46:25.490799 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.480336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.30.104.193:43753)
0809 00:46:25.480821 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.480825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.480834 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:46:25.480869 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.480871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.480883 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.480989 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:25.481219 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.482366 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.484812 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.485572 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.485653 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.485718 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.490654 (+ 4936us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.490720 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":78,"threads_started":1}
I20260809 00:46:25.491247 25525 tablet_bootstrap.cc:654] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:25.491256 25526 tablet_bootstrap.cc:654] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:25.491552 25008 tablet_service.cc:1467] Processing CreateTablet for tablet bd884919aec342728beb9db3cf149863 (DEFAULT_TABLE table=test-table [id=c4001f325b894c2e939cff3558b4afca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:25.491894 25008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd884919aec342728beb9db3cf149863. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:25.491963 25526 log.cc:824] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:25.491963 25525 log.cc:824] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:25.492803 25526 tablet_bootstrap.cc:492] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: No bootstrap required, opened a new log
I20260809 00:46:25.492897 25526 ts_tablet_manager.cc:1397] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:46:25.493155 25525 tablet_bootstrap.cc:492] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: No bootstrap required, opened a new log
I20260809 00:46:25.493255 25525 ts_tablet_manager.cc:1397] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 00:46:25.494905 25525 raft_consensus.cc:348] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.495100 25525 raft_consensus.cc:374] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:25.495151 25525 raft_consensus.cc:729] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b0a8aa92ef04ecbaa6bc40fbe0ec26c, State: Initialized, Role: FOLLOWER
I20260809 00:46:25.495152 25526 raft_consensus.cc:348] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.495275 25525 consensus_queue.cc:260] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.495365 25526 raft_consensus.cc:374] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:25.495415 25526 raft_consensus.cc:729] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf091ebd96654b119c34e13cf4856b47, State: Initialized, Role: FOLLOWER
I20260809 00:46:25.495601 25526 consensus_queue.cc:260] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.495669 25501 heartbeater.cc:503] Master 127.30.104.254:42391 was elected leader, sending a full tablet report...
I20260809 00:46:25.495667 25525 ts_tablet_manager.cc:1428] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 00:46:25.496017 25526 ts_tablet_manager.cc:1428] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:46:25.496017 25303 heartbeater.cc:503] Master 127.30.104.254:42391 was elected leader, sending a full tablet report...
I20260809 00:46:25.496250 25530 tablet_bootstrap.cc:492] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91: Bootstrap starting.
I20260809 00:46:25.497181 25530 tablet_bootstrap.cc:654] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:25.497685 25530 log.cc:824] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:25.498407 25530 tablet_bootstrap.cc:492] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91: No bootstrap required, opened a new log
I20260809 00:46:25.498462 25530 ts_tablet_manager.cc:1397] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:46:25.499945 25530 raft_consensus.cc:348] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.500111 25530 raft_consensus.cc:374] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:25.500136 25530 raft_consensus.cc:729] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaef25430a9545de88642ba6b610ee91, State: Initialized, Role: FOLLOWER
I20260809 00:46:25.500262 25530 consensus_queue.cc:260] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.500532 25530 ts_tablet_manager.cc:1428] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:46:25.500615 25100 heartbeater.cc:503] Master 127.30.104.254:42391 was elected leader, sending a full tablet report...
W20260809 00:46:25.527199 25304 tablet.cc:2369] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:46:25.604184 25101 tablet.cc:2369] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:46:25.699580 25502 tablet.cc:2369] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:46:25.711508 25531 raft_consensus.cc:482] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:25.711647 25531 raft_consensus.cc:504] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.711966 25531 leader_election.cc:288] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaef25430a9545de88642ba6b610ee91 (127.30.104.193:43753), 5b0a8aa92ef04ecbaa6bc40fbe0ec26c (127.30.104.195:43305)
I20260809 00:46:25.714884 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.712287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:41297 (local address 127.30.104.193:43753)
0809 00:46:25.712350 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.712352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.712367 (+ 15us) server_negotiation.cc:407] Connection header received
0809 00:46:25.712401 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.712403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.712413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.712512 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:25.712690 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.713181 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.714137 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.714496 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.714598 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.714607 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.714771 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.714810 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:46:25.714884 25306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.712199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:43753 (local address 127.30.104.194:41297)
0809 00:46:25.712263 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.712277 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:25.712326 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:25.712519 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:25.712521 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:25.712523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:25.712599 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.712601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.713272 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.713273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.714025 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.714027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.714602 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.714603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.714692 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.714695 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:25.714764 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 00:46:25.714807 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 00:46:25.715222 25535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.712335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:43305 (local address 127.30.104.194:46107)
0809 00:46:25.712559 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.712570 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:25.712629 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:25.712796 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:25.712797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:25.712799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:25.712878 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.712879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.713504 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.713505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.714320 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:25.714323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.714930 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.714931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:25.715008 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.715013 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:46:25.715091 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 00:46:25.715136 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":131,"threads_started":1}
I20260809 00:46:25.715315 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.712508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:46107 (local address 127.30.104.195:43305)
0809 00:46:25.712570 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.712573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.712619 (+ 46us) server_negotiation.cc:407] Connection header received
0809 00:46:25.712720 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.712721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.712727 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.712793 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:25.712950 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.713401 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.714444 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.714839 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.714930 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.714938 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.715184 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.715235 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:46:25.715327 25037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd884919aec342728beb9db3cf149863" candidate_uuid: "cf091ebd96654b119c34e13cf4856b47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaef25430a9545de88642ba6b610ee91" is_pre_election: true
I20260809 00:46:25.715583 25037 raft_consensus.cc:2454] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf091ebd96654b119c34e13cf4856b47 in term 0.
I20260809 00:46:25.715869 25133 leader_election.cc:302] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaef25430a9545de88642ba6b610ee91, cf091ebd96654b119c34e13cf4856b47; no voters:
I20260809 00:46:25.716014 25430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd884919aec342728beb9db3cf149863" candidate_uuid: "cf091ebd96654b119c34e13cf4856b47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" is_pre_election: true
I20260809 00:46:25.716053 25531 raft_consensus.cc:2781] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:46:25.716138 25531 raft_consensus.cc:482] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:46:25.716176 25531 raft_consensus.cc:3037] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:25.716176 25430 raft_consensus.cc:2454] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf091ebd96654b119c34e13cf4856b47 in term 0.
I20260809 00:46:25.716971 25531 raft_consensus.cc:504] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.717133 25531 leader_election.cc:288] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [CANDIDATE]: Term 1 election: Requested vote from peers aaef25430a9545de88642ba6b610ee91 (127.30.104.193:43753), 5b0a8aa92ef04ecbaa6bc40fbe0ec26c (127.30.104.195:43305)
I20260809 00:46:25.717345 25037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd884919aec342728beb9db3cf149863" candidate_uuid: "cf091ebd96654b119c34e13cf4856b47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaef25430a9545de88642ba6b610ee91"
I20260809 00:46:25.717360 25430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd884919aec342728beb9db3cf149863" candidate_uuid: "cf091ebd96654b119c34e13cf4856b47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c"
I20260809 00:46:25.717439 25430 raft_consensus.cc:3037] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:25.717444 25037 raft_consensus.cc:3037] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:25.718410 25430 raft_consensus.cc:2454] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf091ebd96654b119c34e13cf4856b47 in term 1.
I20260809 00:46:25.718411 25037 raft_consensus.cc:2454] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf091ebd96654b119c34e13cf4856b47 in term 1.
I20260809 00:46:25.718626 25133 leader_election.cc:302] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aaef25430a9545de88642ba6b610ee91, cf091ebd96654b119c34e13cf4856b47; no voters:
I20260809 00:46:25.718720 25531 raft_consensus.cc:2781] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:25.718925 25531 raft_consensus.cc:686] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 1 LEADER]: Becoming Leader. State: Replica: cf091ebd96654b119c34e13cf4856b47, State: Running, Role: LEADER
I20260809 00:46:25.719033 25531 consensus_queue.cc:237] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } }
I20260809 00:46:25.720003 24825 catalog_manager.cc:5495] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 reported cstate change: term changed from 0 to 1, leader changed from <none> to cf091ebd96654b119c34e13cf4856b47 (127.30.104.194). New cstate: current_term: 1 leader_uuid: "cf091ebd96654b119c34e13cf4856b47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf091ebd96654b119c34e13cf4856b47" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 36219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 } health_report { overall_health: UNKNOWN } } }
I20260809 00:46:25.770782 25522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.768339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.30.104.194:36219)
0809 00:46:25.768439 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.768441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.768449 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:46:25.768493 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.768494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.768512 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.768582 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:46:25.768808 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.769305 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.770054 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.770250 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.770316 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.770476 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:46:25.770569 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:46:25.770652 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.770706 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.770717 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:46:25.772866 25430 raft_consensus.cc:1261] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 1 FOLLOWER]: Refusing update from remote peer cf091ebd96654b119c34e13cf4856b47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:25.772910 25037 raft_consensus.cc:1261] T bd884919aec342728beb9db3cf149863 P aaef25430a9545de88642ba6b610ee91 [term 1 FOLLOWER]: Refusing update from remote peer cf091ebd96654b119c34e13cf4856b47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:25.773340 25531 consensus_queue.cc:1035] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaef25430a9545de88642ba6b610ee91" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 43753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:25.773535 25537 consensus_queue.cc:1035] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b0a8aa92ef04ecbaa6bc40fbe0ec26c" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 43305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:25.776342 25543 mvcc.cc:205] Tried to move back new op lower bound from 7316424236122640384 to 7316424235905736704. Current Snapshot: MvccSnapshot[applied={T|T < 7316424236122640384}]
I20260809 00:46:25.779192 24825 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:25.779301 24825 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:25.780185 24825 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f16dffa105864f17be33317279428d3d: Sending DeleteTablet for 3 replicas of tablet bd884919aec342728beb9db3cf149863
I20260809 00:46:25.780607 25206 tablet_service.cc:1514] Processing DeleteTablet for tablet bd884919aec342728beb9db3cf149863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:25 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:57324
I20260809 00:46:25.780608 25008 tablet_service.cc:1514] Processing DeleteTablet for tablet bd884919aec342728beb9db3cf149863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:25 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:33596
I20260809 00:46:25.780894 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 24911
I20260809 00:46:25.780922 25402 tablet_service.cc:1514] Processing DeleteTablet for tablet bd884919aec342728beb9db3cf149863 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:25 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:56370
I20260809 00:46:25.785391 25564 tablet_replica.cc:330] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: stopping tablet replica
I20260809 00:46:25.785543 25564 raft_consensus.cc:2229] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:25.785678 25564 raft_consensus.cc:2258] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:25.786430 25564 ts_tablet_manager.cc:1905] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:25.788232 25564 ts_tablet_manager.cc:1918] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:25.788321 25564 log.cc:1196] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/bd884919aec342728beb9db3cf149863
I20260809 00:46:25.788609 25564 ts_tablet_manager.cc:1939] T bd884919aec342728beb9db3cf149863 P 5b0a8aa92ef04ecbaa6bc40fbe0ec26c: Deleting consensus metadata
I20260809 00:46:25.789156 24802 catalog_manager.cc:4841] TS 5b0a8aa92ef04ecbaa6bc40fbe0ec26c (127.30.104.195:43305): tablet bd884919aec342728beb9db3cf149863 (table test-table [id=c4001f325b894c2e939cff3558b4afca]) successfully deleted
I20260809 00:46:25.794546 25562 tablet_replica.cc:330] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: stopping tablet replica
I20260809 00:46:25.794818 25562 raft_consensus.cc:2229] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:46:25.795105 25562 raft_consensus.cc:2258] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:25.795710 25562 ts_tablet_manager.cc:1905] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:25.797704 25562 ts_tablet_manager.cc:1918] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:25.797793 25562 log.cc:1196] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/bd884919aec342728beb9db3cf149863
I20260809 00:46:25.798074 25562 ts_tablet_manager.cc:1939] T bd884919aec342728beb9db3cf149863 P cf091ebd96654b119c34e13cf4856b47: Deleting consensus metadata
I20260809 00:46:25.798617 24803 catalog_manager.cc:4841] TS cf091ebd96654b119c34e13cf4856b47 (127.30.104.194:36219): tablet bd884919aec342728beb9db3cf149863 (table test-table [id=c4001f325b894c2e939cff3558b4afca]) successfully deleted
W20260809 00:46:25.800354 24802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:43753 (error 108)
I20260809 00:46:25.800583 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 25111
I20260809 00:46:25.802778 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.800829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:43753 (local address 127.0.0.1:33604)
0809 00:46:25.802671 (+ 1842us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:25.802731 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:43753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1813}
W20260809 00:46:25.802942 24802 catalog_manager.cc:4584] TS aaef25430a9545de88642ba6b610ee91 (127.30.104.193:43753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd884919aec342728beb9db3cf149863: Network error: Client connection negotiation failed: client connection to 127.30.104.193:43753: connect: Connection refused (error 111)
I20260809 00:46:25.811896 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 25310
I20260809 00:46:25.823843 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 24779
2026-08-09T00:46:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2442 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T00:46:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:25Z Disabled control of system clock
I20260809 00:46:25.860374 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:38709
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:38709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:46:25.946892 25575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:25.947165 25575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:25.947202 25575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:25.948458 25575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:25.948537 25575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:25.948559 25575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:25.948581 25575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:25.948611 25575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:25.950045 25575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:38709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:38709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:25.950338 25575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:25.950662 25575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:25.956030 25583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:25.956104 25586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:25.956147 25575 server_base.cc:1034] running on GCE node
W20260809 00:46:25.956061 25582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:25.956460 25575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:25.956764 25575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:25.957958 25575 hybrid_clock.cc:648] HybridClock initialized: now 1786236385957949 us; error 38 us; skew 500 ppm
I20260809 00:46:25.959309 25575 webserver.cc:460] Webserver started at http://127.30.104.254:35261/ using document root <none> and password file <none>
I20260809 00:46:25.959645 25575 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:25.959719 25575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:25.959877 25575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:25.960990 25575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "449cb35d8fc749d09e4772ba9365253f"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "a9c1246015484e473ab11b574a798f91"
server_key_iv: "b7300c89ad814f81bdf987c643effadd"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.961372 25575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "449cb35d8fc749d09e4772ba9365253f"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "a9c1246015484e473ab11b574a798f91"
server_key_iv: "b7300c89ad814f81bdf987c643effadd"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.962746 25575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:25.963541 25594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:25.963729 25575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:25.963836 25575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "449cb35d8fc749d09e4772ba9365253f"
format_stamp: "Formatted at 2026-08-09 00:46:25 on dist-test-slave-c12x"
server_key: "a9c1246015484e473ab11b574a798f91"
server_key_iv: "b7300c89ad814f81bdf987c643effadd"
server_key_version: "encryptionkey@0"
I20260809 00:46:25.963927 25575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:25.978811 25575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:25.979180 25575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:25.979346 25575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:25.983234 25575 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:38709
I20260809 00:46:25.983278 25677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:38709 every 8 connection(s)
I20260809 00:46:25.983727 25575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:25.984802 25678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:25.985379 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 25575
I20260809 00:46:25.985504 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:25.985709 31139 external_mini_cluster.cc:1428] Setting key 83eb0e4a3f62646d109b317d6053a5bb
I20260809 00:46:25.988111 25678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f: Bootstrap starting.
I20260809 00:46:25.988799 25678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:25.989147 25678 log.cc:824] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:25.989939 25678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f: No bootstrap required, opened a new log
I20260809 00:46:25.991645 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:25.986296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48182 (local address 127.30.104.254:38709)
0809 00:46:25.986497 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:25.986504 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:25.986531 (+ 27us) server_negotiation.cc:407] Connection header received
0809 00:46:25.986954 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:25.986960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:25.987019 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:25.987158 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:25.987510 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.988629 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.989687 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:25.990147 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:25.990239 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:25.990935 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:25.990939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:25.990939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:25.991120 (+ 181us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:25.991152 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:25.991162 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:25.991213 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:25.991322 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 00:46:25.991410 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260809 00:46:25.992317 25678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449cb35d8fc749d09e4772ba9365253f" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38709 } }
I20260809 00:46:25.992455 25678 raft_consensus.cc:374] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:25.992476 25678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 449cb35d8fc749d09e4772ba9365253f, State: Initialized, Role: FOLLOWER
I20260809 00:46:25.992616 25678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449cb35d8fc749d09e4772ba9365253f" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38709 } }
I20260809 00:46:25.992692 25678 raft_consensus.cc:388] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:25.992735 25678 raft_consensus.cc:482] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:25.992784 25678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:25.993369 25678 raft_consensus.cc:504] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449cb35d8fc749d09e4772ba9365253f" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38709 } }
I20260809 00:46:25.993458 25678 leader_election.cc:302] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 449cb35d8fc749d09e4772ba9365253f; no voters:
I20260809 00:46:25.993700 25678 leader_election.cc:288] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:25.993784 25685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:25.993945 25685 raft_consensus.cc:686] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [term 1 LEADER]: Becoming Leader. State: Replica: 449cb35d8fc749d09e4772ba9365253f, State: Running, Role: LEADER
I20260809 00:46:25.993988 25678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:25.994107 25685 consensus_queue.cc:237] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449cb35d8fc749d09e4772ba9365253f" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38709 } }
I20260809 00:46:25.994478 25687 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 449cb35d8fc749d09e4772ba9365253f. Latest consensus state: current_term: 1 leader_uuid: "449cb35d8fc749d09e4772ba9365253f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449cb35d8fc749d09e4772ba9365253f" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38709 } } }
I20260809 00:46:25.994607 25687 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:25.994479 25686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "449cb35d8fc749d09e4772ba9365253f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "449cb35d8fc749d09e4772ba9365253f" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38709 } } }
I20260809 00:46:25.994786 25686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:25.994987 25695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:25.995577 25695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:25.997114 25695 catalog_manager.cc:1347] Generated new cluster ID: b3bd3208bd3f4e17ac9f4a543e5229a8
I20260809 00:46:25.997169 25695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:26.004709 25695 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:26.005247 25695 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:26.008410 25695 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f: Generated new TSK 0
I20260809 00:46:26.008613 25695 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:26.019778 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:38709
--builtin_ntp_servers=127.30.104.212:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:46:26.106379 25709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:26.106644 25709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:26.106691 25709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:26.107913 25709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:26.107982 25709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:26.108024 25709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:46:26.109452 25709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:38709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:26.109766 25709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:26.110107 25709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:26.115669 25715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:26.115869 25716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:26.115664 25720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:26.115785 25709 server_base.cc:1034] running on GCE node
I20260809 00:46:26.116381 25709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:26.116714 25709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:26.117910 25709 hybrid_clock.cc:648] HybridClock initialized: now 1786236386117890 us; error 36 us; skew 500 ppm
I20260809 00:46:26.119513 25709 webserver.cc:460] Webserver started at http://127.30.104.193:45699/ using document root <none> and password file <none>
I20260809 00:46:26.119801 25709 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:26.119850 25709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:26.120030 25709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:26.121173 25709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "f9e37dcbc93844849bfc47d81033fdff"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "f660c1ee7c1a4f861a48711c050ed1b5"
server_key_iv: "db4dba8273f64f25db46ba63fa4f2277"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.121596 25709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "f9e37dcbc93844849bfc47d81033fdff"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "f660c1ee7c1a4f861a48711c050ed1b5"
server_key_iv: "db4dba8273f64f25db46ba63fa4f2277"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.123001 25709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:26.123872 25727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.124043 25709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:46:26.124162 25709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "f9e37dcbc93844849bfc47d81033fdff"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "f660c1ee7c1a4f861a48711c050ed1b5"
server_key_iv: "db4dba8273f64f25db46ba63fa4f2277"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.124243 25709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:26.137231 25709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:26.137593 25709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:26.137760 25709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:26.138036 25709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:26.138415 25709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:26.138453 25709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.138475 25709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:26.138514 25709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.144793 25709 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:38549
I20260809 00:46:26.144904 25897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:38549 every 8 connection(s)
I20260809 00:46:26.145255 25709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:46:26.146574 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 25709
I20260809 00:46:26.146731 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:46:26.146934 31139 external_mini_cluster.cc:1428] Setting key dc4aebc4563065ac30625b362f24fb9f
I20260809 00:46:26.148062 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:38709
--builtin_ntp_servers=127.30.104.212:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:46:26.151238 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.146074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:57123 (local address 127.30.104.254:38709)
0809 00:46:26.146121 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.146123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.146389 (+ 266us) server_negotiation.cc:407] Connection header received
0809 00:46:26.147034 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.147036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.147055 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.147118 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:26.148007 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.148524 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.149584 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.149793 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.149882 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.150886 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:26.150889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:26.150890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:26.150918 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:26.150933 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:26.150937 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:26.151000 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.151114 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.151151 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:46:26.151563 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.146048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:38709 (local address 127.30.104.193:57123)
0809 00:46:26.146253 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.146288 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:26.146914 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:26.147193 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:26.147197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:26.147208 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:26.147896 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.147902 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.148635 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.148636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.149482 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.149484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.149892 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.149894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.150050 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.150663 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:26.150663 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:26.150707 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:26.150735 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:26.151014 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:26.151021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:26.151024 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:26.151117 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 00:46:26.151233 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260809 00:46:26.152274 25898 heartbeater.cc:347] Connected to a master server at 127.30.104.254:38709
I20260809 00:46:26.152379 25898 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:26.152590 25898 heartbeater.cc:511] Master 127.30.104.254:38709 requested a full tablet report, sending...
I20260809 00:46:26.153131 25622 ts_manager.cc:194] Registered new tserver with Master: f9e37dcbc93844849bfc47d81033fdff (127.30.104.193:38549)
I20260809 00:46:26.153772 25622 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.193:57123
W20260809 00:46:26.239293 25904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:26.239625 25904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:26.239689 25904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:26.241008 25904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:26.241089 25904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:26.241142 25904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:46:26.242651 25904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:38709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:26.242955 25904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:26.243314 25904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:26.249116 25915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:26.249130 25911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:26.249110 25912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:26.249294 25904 server_base.cc:1034] running on GCE node
I20260809 00:46:26.249567 25904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:26.249840 25904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:26.251000 25904 hybrid_clock.cc:648] HybridClock initialized: now 1786236386250996 us; error 39 us; skew 500 ppm
I20260809 00:46:26.252594 25904 webserver.cc:460] Webserver started at http://127.30.104.194:35227/ using document root <none> and password file <none>
I20260809 00:46:26.252928 25904 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:26.253013 25904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:26.253171 25904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:26.254547 25904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "eb01ed1015c74d00bc99ac1d02dedf99"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "45c9b453f868d60e965f1a788274f6ec"
server_key_iv: "8e6b84245ce8f981e7c0043508a94950"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.254971 25904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "eb01ed1015c74d00bc99ac1d02dedf99"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "45c9b453f868d60e965f1a788274f6ec"
server_key_iv: "8e6b84245ce8f981e7c0043508a94950"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.256505 25904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260809 00:46:26.257647 25922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.258019 25904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:26.258147 25904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "eb01ed1015c74d00bc99ac1d02dedf99"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "45c9b453f868d60e965f1a788274f6ec"
server_key_iv: "8e6b84245ce8f981e7c0043508a94950"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.258239 25904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:26.277364 25904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:26.278075 25904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:26.278276 25904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:26.278774 25904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:26.279644 25904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:26.279726 25904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.279770 25904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:26.279815 25904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.286489 25904 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:34959
I20260809 00:46:26.286528 26105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:34959 every 8 connection(s)
I20260809 00:46:26.286919 25904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:46:26.292796 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.287761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:33265 (local address 127.30.104.254:38709)
0809 00:46:26.287808 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.287810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.288098 (+ 288us) server_negotiation.cc:407] Connection header received
0809 00:46:26.288779 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.288780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.288792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.288855 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:26.289587 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.290128 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.291266 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.291485 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.291564 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.292427 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:26.292428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:26.292429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:26.292433 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:26.292448 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:26.292452 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:26.292506 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.292646 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.292731 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:46:26.292853 26108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.287774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:38709 (local address 127.30.104.194:33265)
0809 00:46:26.287952 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.287988 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:26.288637 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:26.288912 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:26.288916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:26.288924 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:26.289481 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.289485 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.290223 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.290224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.291157 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.291160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.291576 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.291577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.291701 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.292276 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:26.292276 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:26.292311 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:26.292338 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:26.292518 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:26.292523 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:26.292526 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:26.292582 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:46:26.292675 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260809 00:46:26.293439 26106 heartbeater.cc:347] Connected to a master server at 127.30.104.254:38709
I20260809 00:46:26.293578 26106 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:26.293833 26106 heartbeater.cc:511] Master 127.30.104.254:38709 requested a full tablet report, sending...
I20260809 00:46:26.294410 25622 ts_manager.cc:194] Registered new tserver with Master: eb01ed1015c74d00bc99ac1d02dedf99 (127.30.104.194:34959)
I20260809 00:46:26.294838 25622 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.194:33265
I20260809 00:46:26.295775 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 25904
I20260809 00:46:26.295862 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:46:26.296008 31139 external_mini_cluster.cc:1428] Setting key 6fe39e79d242fc24bc753052a85edcc6
I20260809 00:46:26.297029 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:38709
--builtin_ntp_servers=127.30.104.212:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:46:26.381721 26109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:26.381937 26109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:26.381959 26109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:26.383139 26109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:26.383184 26109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:26.383210 26109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:46:26.384688 26109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:38709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:26.384951 26109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:26.385272 26109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:26.390672 26117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:26.390733 26109 server_base.cc:1034] running on GCE node
W20260809 00:46:26.390672 26115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:26.390712 26121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:26.391098 26109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:26.391299 26109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:26.392441 26109 hybrid_clock.cc:648] HybridClock initialized: now 1786236386392414 us; error 41 us; skew 500 ppm
I20260809 00:46:26.393716 26109 webserver.cc:460] Webserver started at http://127.30.104.195:34755/ using document root <none> and password file <none>
I20260809 00:46:26.393972 26109 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:26.394019 26109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:26.394230 26109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:26.395494 26109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "6c9db92f5fd74a0591ad8ade314492e9"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "8aa56f10b4f6dd6d64189f3a34a1fe6f"
server_key_iv: "4b4a9c35a356e8336622bb8f15153e84"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.395893 26109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "6c9db92f5fd74a0591ad8ade314492e9"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "8aa56f10b4f6dd6d64189f3a34a1fe6f"
server_key_iv: "4b4a9c35a356e8336622bb8f15153e84"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.397231 26109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:26.398001 26128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.398175 26109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:46:26.398283 26109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "6c9db92f5fd74a0591ad8ade314492e9"
format_stamp: "Formatted at 2026-08-09 00:46:26 on dist-test-slave-c12x"
server_key: "8aa56f10b4f6dd6d64189f3a34a1fe6f"
server_key_iv: "4b4a9c35a356e8336622bb8f15153e84"
server_key_version: "encryptionkey@0"
I20260809 00:46:26.398360 26109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:26.414688 26109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:26.415031 26109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:26.415191 26109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:26.415494 26109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:26.415868 26109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:26.415925 26109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.415961 26109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:26.416004 26109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:26.421716 26109 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:35849
I20260809 00:46:26.421764 26311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:35849 every 8 connection(s)
I20260809 00:46:26.422107 26109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:46:26.423413 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 26109
I20260809 00:46:26.423600 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:46:26.423831 31139 external_mini_cluster.cc:1428] Setting key a08f453a9edcf7474e32b5101e8bd445
I20260809 00:46:26.427640 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.422888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:47955 (local address 127.30.104.254:38709)
0809 00:46:26.422961 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.422963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.423177 (+ 214us) server_negotiation.cc:407] Connection header received
0809 00:46:26.423779 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.423781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.423795 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.423883 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:26.424607 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.425124 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.426110 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.426376 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.426471 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.427290 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:26.427292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:26.427292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:26.427297 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:26.427311 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:26.427316 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:26.427366 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.427527 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.427555 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:46:26.427889 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.422863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:38709 (local address 127.30.104.195:47955)
0809 00:46:26.423043 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.423086 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:26.423650 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:26.423970 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:26.423982 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:26.423990 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:26.424500 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.424505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.425229 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.425231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.426015 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.426017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.426483 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.426484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.426621 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.427137 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:26.427137 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:26.427167 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:26.427190 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:26.427373 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:26.427377 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:26.427380 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:26.427559 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 00:46:26.427668 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260809 00:46:26.428378 26315 heartbeater.cc:347] Connected to a master server at 127.30.104.254:38709
I20260809 00:46:26.428467 26315 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:26.428669 26315 heartbeater.cc:511] Master 127.30.104.254:38709 requested a full tablet report, sending...
I20260809 00:46:26.429067 25622 ts_manager.cc:194] Registered new tserver with Master: 6c9db92f5fd74a0591ad8ade314492e9 (127.30.104.195:35849)
I20260809 00:46:26.429428 25622 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.195:47955
I20260809 00:46:26.435683 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:46:26.440402 25683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.437156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35102 (local address 127.30.104.254:38709)
0809 00:46:26.437238 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.437240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.437352 (+ 112us) server_negotiation.cc:407] Connection header received
0809 00:46:26.437434 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.437435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.437444 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.437495 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:26.437694 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.438189 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.438985 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.439207 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.439286 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.440048 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:26.440050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:26.440050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:26.440056 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:26.440073 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:26.440077 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:26.440210 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.440294 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.440307 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:46:26.442131 25622 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:46:26.443877 25622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:46:26.450503 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.446052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:35849 (local address 127.0.0.1:52028)
0809 00:46:26.446257 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.446300 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:26.446372 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:26.446639 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:26.446642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:26.446645 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:26.446720 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.446722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.447823 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.447824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.449170 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.449174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.450240 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.450241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.450308 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.450311 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:26.450372 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:46:26.450405 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":67,"threads_started":1}
I20260809 00:46:26.450539 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.446166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52028 (local address 127.30.104.195:35849)
0809 00:46:26.446391 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.446412 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.446445 (+ 33us) server_negotiation.cc:407] Connection header received
0809 00:46:26.446491 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.446494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.446512 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.446608 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:26.446793 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.447740 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.449274 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.450030 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.450104 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.450204 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.450408 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.450453 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":98,"threads_started":1}
I20260809 00:46:26.450503 26325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.446063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:34959 (local address 127.0.0.1:52836)
0809 00:46:26.446352 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.446362 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:26.446404 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:26.447011 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:26.447012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:26.447014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:26.447051 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.447053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.448089 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.448090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.449011 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.449014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.450126 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.450128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.450219 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.450237 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 00:46:26.450289 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 00:46:26.450350 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":9,"thread_start_us":107,"threads_started":1}
I20260809 00:46:26.451014 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.446130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52836 (local address 127.30.104.194:34959)
0809 00:46:26.446851 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.446856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.446867 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:46:26.446913 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.446916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.446933 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.447016 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:26.447116 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.447955 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.449147 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.450003 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.450580 (+ 577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.450826 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.450850 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.450917 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":109,"threads_started":1}
I20260809 00:46:26.451581 26333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.446279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:38549 (local address 127.0.0.1:50268)
0809 00:46:26.446704 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.446717 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:26.446782 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:26.448009 (+ 1227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:26.448010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:26.448013 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:26.448064 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.448067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.449295 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.449296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.449957 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.449958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.451316 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.451317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.451366 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.451369 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:26.451436 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 00:46:26.451504 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":70,"threads_started":1}
I20260809 00:46:26.451454 26196 tablet_service.cc:1467] Processing CreateTablet for tablet d6dec3be30814ea88928aa52542d8127 (DEFAULT_TABLE table=test-table [id=44473b82940949d7a57e849b8b2554d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:26.451679 25999 tablet_service.cc:1467] Processing CreateTablet for tablet d6dec3be30814ea88928aa52542d8127 (DEFAULT_TABLE table=test-table [id=44473b82940949d7a57e849b8b2554d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:26.451601 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.446494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.30.104.193:38549)
0809 00:46:26.446697 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.446701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.447845 (+ 1144us) server_negotiation.cc:407] Connection header received
0809 00:46:26.447893 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.447895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.447911 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.447988 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:26.448137 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.449211 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.450070 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.451135 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.451207 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.451284 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.451487 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.451524 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260809 00:46:26.451923 26196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6dec3be30814ea88928aa52542d8127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:26.452018 25999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6dec3be30814ea88928aa52542d8127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:26.452442 25805 tablet_service.cc:1467] Processing CreateTablet for tablet d6dec3be30814ea88928aa52542d8127 (DEFAULT_TABLE table=test-table [id=44473b82940949d7a57e849b8b2554d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:26.452775 25805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6dec3be30814ea88928aa52542d8127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:26.455312 26335 tablet_bootstrap.cc:492] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff: Bootstrap starting.
I20260809 00:46:26.455794 26337 tablet_bootstrap.cc:492] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Bootstrap starting.
I20260809 00:46:26.456194 26335 tablet_bootstrap.cc:654] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:26.456511 26335 log.cc:824] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:26.456554 26337 tablet_bootstrap.cc:654] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:26.456884 26337 log.cc:824] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:26.457245 26336 tablet_bootstrap.cc:492] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: Bootstrap starting.
I20260809 00:46:26.457345 26335 tablet_bootstrap.cc:492] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff: No bootstrap required, opened a new log
I20260809 00:46:26.457424 26335 ts_tablet_manager.cc:1397] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:46:26.457677 26337 tablet_bootstrap.cc:492] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: No bootstrap required, opened a new log
I20260809 00:46:26.457748 26337 ts_tablet_manager.cc:1397] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:46:26.457978 26336 tablet_bootstrap.cc:654] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:26.458290 26336 log.cc:824] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:26.459276 26336 tablet_bootstrap.cc:492] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: No bootstrap required, opened a new log
I20260809 00:46:26.459376 26336 ts_tablet_manager.cc:1397] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:46:26.460186 26335 raft_consensus.cc:348] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.460382 26335 raft_consensus.cc:374] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:26.460421 26335 raft_consensus.cc:729] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9e37dcbc93844849bfc47d81033fdff, State: Initialized, Role: FOLLOWER
I20260809 00:46:26.460356 26337 raft_consensus.cc:348] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.460494 26337 raft_consensus.cc:374] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:26.460544 26337 raft_consensus.cc:729] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb01ed1015c74d00bc99ac1d02dedf99, State: Initialized, Role: FOLLOWER
I20260809 00:46:26.460584 26335 consensus_queue.cc:260] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.460672 26337 consensus_queue.cc:260] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.460879 26335 ts_tablet_manager.cc:1428] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:46:26.461015 25898 heartbeater.cc:503] Master 127.30.104.254:38709 was elected leader, sending a full tablet report...
I20260809 00:46:26.461196 26106 heartbeater.cc:503] Master 127.30.104.254:38709 was elected leader, sending a full tablet report...
I20260809 00:46:26.460951 26337 ts_tablet_manager.cc:1428] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:46:26.462257 26336 raft_consensus.cc:348] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.462452 26336 raft_consensus.cc:374] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:26.462495 26336 raft_consensus.cc:729] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c9db92f5fd74a0591ad8ade314492e9, State: Initialized, Role: FOLLOWER
I20260809 00:46:26.462625 26336 consensus_queue.cc:260] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.462886 26336 ts_tablet_manager.cc:1428] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:46:26.462968 26315 heartbeater.cc:503] Master 127.30.104.254:38709 was elected leader, sending a full tablet report...
W20260809 00:46:26.537562 26107 tablet.cc:2369] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:46:26.578395 26344 raft_consensus.cc:482] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:26.578531 26344 raft_consensus.cc:504] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.578918 26344 leader_election.cc:288] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb01ed1015c74d00bc99ac1d02dedf99 (127.30.104.194:34959), f9e37dcbc93844849bfc47d81033fdff (127.30.104.193:38549)
I20260809 00:46:26.583211 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.579229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:59897 (local address 127.30.104.193:38549)
0809 00:46:26.579284 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.579286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.579296 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:46:26.579334 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.579335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.579361 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.579504 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:26.579722 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.580601 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.582236 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.582817 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.582897 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.582906 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.583095 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.583138 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:46:26.583211 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.579327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:58249 (local address 127.30.104.194:34959)
0809 00:46:26.579377 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.579390 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.579480 (+ 90us) server_negotiation.cc:407] Connection header received
0809 00:46:26.579546 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.579547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.579560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.579615 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:26.579797 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.580631 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.582236 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.582816 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.582898 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.582906 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.583096 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.583137 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:46:26.583209 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.579148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:38549 (local address 127.30.104.195:59897)
0809 00:46:26.579203 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.579217 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:26.579271 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:26.579495 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:26.579496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:26.579499 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:26.579630 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.579632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.580746 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.580747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.582118 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.582122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.582909 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.582911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.582991 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.582994 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:26.583058 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:46:26.583107 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":65}
I20260809 00:46:26.583221 26345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.579166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:34959 (local address 127.30.104.195:58249)
0809 00:46:26.579388 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.579396 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:26.579492 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:26.579634 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:26.579635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:26.579636 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:26.579709 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.579711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.580746 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.580747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.582118 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:26.582122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.582924 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.582925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:26.582990 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.582994 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:46:26.583068 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 00:46:26.583106 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":102,"thread_start_us":109,"threads_started":1}
I20260809 00:46:26.583660 25836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6dec3be30814ea88928aa52542d8127" candidate_uuid: "6c9db92f5fd74a0591ad8ade314492e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e37dcbc93844849bfc47d81033fdff" is_pre_election: true
I20260809 00:46:26.583663 26028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6dec3be30814ea88928aa52542d8127" candidate_uuid: "6c9db92f5fd74a0591ad8ade314492e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" is_pre_election: true
I20260809 00:46:26.583859 25836 raft_consensus.cc:2454] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c9db92f5fd74a0591ad8ade314492e9 in term 0.
I20260809 00:46:26.583868 26028 raft_consensus.cc:2454] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c9db92f5fd74a0591ad8ade314492e9 in term 0.
I20260809 00:46:26.584123 26132 leader_election.cc:302] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c9db92f5fd74a0591ad8ade314492e9, f9e37dcbc93844849bfc47d81033fdff; no voters:
I20260809 00:46:26.584328 26344 raft_consensus.cc:2781] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:46:26.584410 26344 raft_consensus.cc:482] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:46:26.584446 26344 raft_consensus.cc:3037] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:26.585209 26344 raft_consensus.cc:504] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.585384 26344 leader_election.cc:288] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [CANDIDATE]: Term 1 election: Requested vote from peers eb01ed1015c74d00bc99ac1d02dedf99 (127.30.104.194:34959), f9e37dcbc93844849bfc47d81033fdff (127.30.104.193:38549)
I20260809 00:46:26.585569 26028 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6dec3be30814ea88928aa52542d8127" candidate_uuid: "6c9db92f5fd74a0591ad8ade314492e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb01ed1015c74d00bc99ac1d02dedf99"
I20260809 00:46:26.585590 25836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6dec3be30814ea88928aa52542d8127" candidate_uuid: "6c9db92f5fd74a0591ad8ade314492e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9e37dcbc93844849bfc47d81033fdff"
I20260809 00:46:26.585662 25836 raft_consensus.cc:3037] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:26.585662 26028 raft_consensus.cc:3037] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:26.586658 25836 raft_consensus.cc:2454] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c9db92f5fd74a0591ad8ade314492e9 in term 1.
I20260809 00:46:26.586678 26028 raft_consensus.cc:2454] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c9db92f5fd74a0591ad8ade314492e9 in term 1.
I20260809 00:46:26.586843 26129 leader_election.cc:302] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c9db92f5fd74a0591ad8ade314492e9, eb01ed1015c74d00bc99ac1d02dedf99; no voters:
I20260809 00:46:26.586961 26344 raft_consensus.cc:2781] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:26.587131 26344 raft_consensus.cc:686] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 1 LEADER]: Becoming Leader. State: Replica: 6c9db92f5fd74a0591ad8ade314492e9, State: Running, Role: LEADER
I20260809 00:46:26.587239 26344 consensus_queue.cc:237] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } }
I20260809 00:46:26.588286 25622 catalog_manager.cc:5495] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c9db92f5fd74a0591ad8ade314492e9 (127.30.104.195). New cstate: current_term: 1 leader_uuid: "6c9db92f5fd74a0591ad8ade314492e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9db92f5fd74a0591ad8ade314492e9" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 } health_report { overall_health: UNKNOWN } } }
I20260809 00:46:26.601730 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.599125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52038 (local address 127.30.104.195:35849)
0809 00:46:26.599180 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:26.599182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:26.599192 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:46:26.599241 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:26.599242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:26.599262 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:26.599331 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:46:26.599607 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.600094 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.600957 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:26.601149 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:26.601203 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:26.601354 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:46:26.601446 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:46:26.601520 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:26.601581 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 00:46:26.601589 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:46:26.603819 25836 raft_consensus.cc:1261] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 1 FOLLOWER]: Refusing update from remote peer 6c9db92f5fd74a0591ad8ade314492e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:26.603819 26028 raft_consensus.cc:1261] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 1 FOLLOWER]: Refusing update from remote peer 6c9db92f5fd74a0591ad8ade314492e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:26.604259 26344 consensus_queue.cc:1035] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9e37dcbc93844849bfc47d81033fdff" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:26.604434 26348 consensus_queue.cc:1035] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb01ed1015c74d00bc99ac1d02dedf99" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:26.610345 25617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:26.610487 25617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:26.611577 25617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 449cb35d8fc749d09e4772ba9365253f: Sending DeleteTablet for 3 replicas of tablet d6dec3be30814ea88928aa52542d8127
I20260809 00:46:26.611936 26196 tablet_service.cc:1514] Processing DeleteTablet for tablet d6dec3be30814ea88928aa52542d8127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:26 UTC) from {username='slave'} at 127.0.0.1:52028
I20260809 00:46:26.612056 25805 tablet_service.cc:1514] Processing DeleteTablet for tablet d6dec3be30814ea88928aa52542d8127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:26 UTC) from {username='slave'} at 127.0.0.1:50268
I20260809 00:46:26.612195 26363 tablet_replica.cc:330] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: stopping tablet replica
I20260809 00:46:26.612300 26363 raft_consensus.cc:2229] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:46:26.612440 26363 raft_consensus.cc:2258] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:26.612557 25999 tablet_service.cc:1514] Processing DeleteTablet for tablet d6dec3be30814ea88928aa52542d8127 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:26 UTC) from {username='slave'} at 127.0.0.1:52836
I20260809 00:46:26.612528 26364 tablet_replica.cc:330] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff: stopping tablet replica
I20260809 00:46:26.612783 26364 raft_consensus.cc:2229] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:26.612967 26364 raft_consensus.cc:2258] T d6dec3be30814ea88928aa52542d8127 P f9e37dcbc93844849bfc47d81033fdff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:26.613113 26363 ts_tablet_manager.cc:1905] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:26.613225 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 25709
I20260809 00:46:26.631932 26365 tablet_replica.cc:330] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: stopping tablet replica
I20260809 00:46:26.632144 26365 raft_consensus.cc:2229] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:26.632301 26365 raft_consensus.cc:2258] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:26.633378 26365 ts_tablet_manager.cc:1905] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:26.634976 26365 ts_tablet_manager.cc:1918] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:26.635134 26365 log.cc:1196] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/d6dec3be30814ea88928aa52542d8127
I20260809 00:46:26.635551 26365 ts_tablet_manager.cc:1939] T d6dec3be30814ea88928aa52542d8127 P eb01ed1015c74d00bc99ac1d02dedf99: Deleting consensus metadata
I20260809 00:46:26.636932 26363 ts_tablet_manager.cc:1918] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:26.637032 26363 log.cc:1196] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/d6dec3be30814ea88928aa52542d8127
I20260809 00:46:26.637319 26363 ts_tablet_manager.cc:1939] T d6dec3be30814ea88928aa52542d8127 P 6c9db92f5fd74a0591ad8ade314492e9: Deleting consensus metadata
I20260809 00:46:26.637885 25602 catalog_manager.cc:4841] TS 6c9db92f5fd74a0591ad8ade314492e9 (127.30.104.195:35849): tablet d6dec3be30814ea88928aa52542d8127 (table test-table [id=44473b82940949d7a57e849b8b2554d2]) successfully deleted
I20260809 00:46:26.639714 25597 catalog_manager.cc:4841] TS eb01ed1015c74d00bc99ac1d02dedf99 (127.30.104.194:34959): tablet d6dec3be30814ea88928aa52542d8127 (table test-table [id=44473b82940949d7a57e849b8b2554d2]) successfully deleted
W20260809 00:46:26.640164 25602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:38549 (error 108)
I20260809 00:46:26.640753 26333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:26.640592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:38549 (local address 127.0.0.1:50274)
0809 00:46:26.640659 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:26.640709 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:38549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 00:46:26.640846 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 25904
W20260809 00:46:26.649355 25602 catalog_manager.cc:4584] TS f9e37dcbc93844849bfc47d81033fdff (127.30.104.193:38549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6dec3be30814ea88928aa52542d8127: Network error: Client connection negotiation failed: client connection to 127.30.104.193:38549: connect: Connection refused (error 111)
I20260809 00:46:26.654090 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 26109
I20260809 00:46:26.666314 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 25575
2026-08-09T00:46:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (844 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:26 dist-test-slave-c12x krb5kdc[26372](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:26 dist-test-slave-c12x krb5kdc[26372](info): set up 2 sockets
Aug 09 00:46:26 dist-test-slave-c12x krb5kdc[26372](info): commencing operation
krb5kdc: starting...
W20260809 00:46:27.903146 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:27 dist-test-slave-c12x krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:28.030369 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44859
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:44859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:28.120002 26388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:28.120280 26388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:28.120321 26388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:28.121569 26388 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:28.121649 26388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:28.121676 26388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:28.121708 26388 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:28.121732 26388 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:28.123643 26388 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:44859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:44859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:28.123982 26388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:28.124245 26388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:28.129551 26406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:28.129516 26403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:28.129516 26401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:28.129678 26388 server_base.cc:1034] running on GCE node
I20260809 00:46:28.130040 26388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:28.130303 26388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:28.131512 26388 hybrid_clock.cc:648] HybridClock initialized: now 1786236388131485 us; error 42 us; skew 500 ppm
Aug 09 00:46:28 dist-test-slave-c12x krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:28.134349 26388 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:28.134852 26388 webserver.cc:460] Webserver started at http://127.30.104.254:44413/ using document root <none> and password file <none>
I20260809 00:46:28.135069 26388 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:28.135113 26388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:28.135208 26388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:28.136351 26388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "4b17081dc1b741b3bc05cf042261a028"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "09f7e106bae48e17530db63ae4615f97"
server_key_iv: "75fb446f2bc703c4da085b828a49df8f"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.136668 26388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "4b17081dc1b741b3bc05cf042261a028"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "09f7e106bae48e17530db63ae4615f97"
server_key_iv: "75fb446f2bc703c4da085b828a49df8f"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.138062 26388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:46:28.138880 26415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.139096 26388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:28.139195 26388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "4b17081dc1b741b3bc05cf042261a028"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "09f7e106bae48e17530db63ae4615f97"
server_key_iv: "75fb446f2bc703c4da085b828a49df8f"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.139259 26388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:28.162868 26388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:28.165416 26388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:28.165556 26388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:28.169929 26388 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:44859
I20260809 00:46:28.170013 26498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:44859 every 8 connection(s)
I20260809 00:46:28.170390 26388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:28.171370 26499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:28.173851 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028: Bootstrap starting.
I20260809 00:46:28.174481 26499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:28.174790 26499 log.cc:824] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:28.175635 26499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028: No bootstrap required, opened a new log
I20260809 00:46:28.176621 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 26388
I20260809 00:46:28.176718 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:28.176959 31139 external_mini_cluster.cc:1428] Setting key 23ddcb2c90cea43d79279c10ce4b75bd
I20260809 00:46:28.177692 26499 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17081dc1b741b3bc05cf042261a028" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44859 } }
I20260809 00:46:28.177912 26499 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:28.177959 26499 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b17081dc1b741b3bc05cf042261a028, State: Initialized, Role: FOLLOWER
I20260809 00:46:28.178097 26499 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17081dc1b741b3bc05cf042261a028" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44859 } }
I20260809 00:46:28.178192 26499 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:28.178223 26499 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:28.178264 26499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:28.179382 26499 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17081dc1b741b3bc05cf042261a028" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44859 } }
I20260809 00:46:28.179522 26499 leader_election.cc:302] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b17081dc1b741b3bc05cf042261a028; no voters:
I20260809 00:46:28.179803 26499 leader_election.cc:288] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:28.179901 26506 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:28.180099 26499 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:28.180101 26506 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [term 1 LEADER]: Becoming Leader. State: Replica: 4b17081dc1b741b3bc05cf042261a028, State: Running, Role: LEADER
I20260809 00:46:28.180467 26506 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17081dc1b741b3bc05cf042261a028" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44859 } }
I20260809 00:46:28.180876 26507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b17081dc1b741b3bc05cf042261a028. Latest consensus state: current_term: 1 leader_uuid: "4b17081dc1b741b3bc05cf042261a028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17081dc1b741b3bc05cf042261a028" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44859 } } }
I20260809 00:46:28.180971 26507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:28.181233 26521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:28.181329 26508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b17081dc1b741b3bc05cf042261a028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b17081dc1b741b3bc05cf042261a028" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 44859 } } }
I20260809 00:46:28.181402 26508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:28.181788 26521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:28.184437 26521 catalog_manager.cc:1347] Generated new cluster ID: f05aea5bf7094d11982e76e3b197ae5a
I20260809 00:46:28.184495 26521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:46:28 dist-test-slave-c12x krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:28.188912 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.177607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49678 (local address 127.30.104.254:44859)
0809 00:46:28.178010 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.178035 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.178117 (+ 82us) server_negotiation.cc:407] Connection header received
0809 00:46:28.178243 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.178251 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.178311 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.178463 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:28.179155 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.180736 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.181924 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.182549 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.182654 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.185494 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:28.185498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:28.185498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:28.185511 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:28.187291 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.187736 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.187740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.187740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.187846 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.188071 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.188072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.188072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.188329 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:28.188519 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.188590 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.188738 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":109,"threads_started":1}
I20260809 00:46:28.193962 26521 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:28.194763 26521 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:28.199994 26521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028: Generated new TSK 0
I20260809 00:46:28.200196 26521 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.104.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
Entry for principal kudu/127.30.104.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab.
I20260809 00:46:28.238358 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44859
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:28.342374 26540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:28.342628 26540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:28.342672 26540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:28.343986 26540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:28.344060 26540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:28.344110 26540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:46:28.345732 26540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.193.keytab
--principal=kudu/127.30.104.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:28.346017 26540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:28.346274 26540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:28.351639 26546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:28.351770 26540 server_base.cc:1034] running on GCE node
W20260809 00:46:28.351631 26550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:28.351635 26547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:28.352109 26540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:28.352367 26540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:28.353559 26540 hybrid_clock.cc:648] HybridClock initialized: now 1786236388353541 us; error 41 us; skew 500 ppm
Aug 09 00:46:28 dist-test-slave-c12x krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:28.356325 26540 init.cc:377] Logged in from keytab as kudu/127.30.104.193@KRBTEST.COM (short username kudu)
I20260809 00:46:28.356812 26540 webserver.cc:460] Webserver started at http://127.30.104.193:41353/ using document root <none> and password file <none>
I20260809 00:46:28.357071 26540 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:28.357149 26540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:28.357301 26540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:28.358389 26540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "0c6d758a477b4a1a986364468a1f217f"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "7e6646c90a7d66cada365fecc5d7ad81"
server_key_iv: "afe7844131a6bc4717f179a0a8f65340"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.358774 26540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "0c6d758a477b4a1a986364468a1f217f"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "7e6646c90a7d66cada365fecc5d7ad81"
server_key_iv: "afe7844131a6bc4717f179a0a8f65340"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.360321 26540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:46:28.361109 26560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.361289 26540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:28.361402 26540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "0c6d758a477b4a1a986364468a1f217f"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "7e6646c90a7d66cada365fecc5d7ad81"
server_key_iv: "afe7844131a6bc4717f179a0a8f65340"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.361483 26540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:28.371681 26540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:28.373805 26540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:28.373979 26540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:28.374239 26540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:28.374576 26540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:28.374636 26540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.374682 26540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:28.374706 26540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.381098 26540 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:46251
I20260809 00:46:28.381143 26729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:46251 every 8 connection(s)
I20260809 00:46:28.381577 26540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:46:28.385808 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 26540
I20260809 00:46:28.385968 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:46:28.386174 31139 external_mini_cluster.cc:1428] Setting key 544c6ce320574ce0f01c75c6effd87ab
Aug 09 00:46:28 dist-test-slave-c12x krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.193@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:28.392465 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.382352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:46723 (local address 127.30.104.254:44859)
0809 00:46:28.382425 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.382429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.382685 (+ 256us) server_negotiation.cc:407] Connection header received
0809 00:46:28.383189 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.383191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.383205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.383269 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:28.384345 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.384871 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.385869 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.386217 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.386296 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.389021 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:28.389023 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:28.389023 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:28.389032 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:28.390588 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.391177 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.391178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.391179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.391256 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.391646 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.391647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.391648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.391825 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:28.391905 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.392306 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.392392 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:46:28.392802 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.382344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44859 (local address 127.30.104.193:46723)
0809 00:46:28.382541 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.382579 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:28.383077 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:28.383349 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:28.383352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:28.384031 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:28.384242 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.384246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.384974 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.384975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.385778 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.385779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.386383 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.386384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.386564 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.387215 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:28.387227 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:28.388913 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:28.390689 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:28.390699 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:28.390702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:28.391058 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:28.391346 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:28.391347 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:28.391347 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:28.391561 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:28.391912 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:28.391921 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:28.392132 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 00:46:28.392254 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 00:46:28.392494 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":116,"threads_started":1}
I20260809 00:46:28.393855 26731 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44859
I20260809 00:46:28.393968 26731 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:28.394186 26731 heartbeater.cc:511] Master 127.30.104.254:44859 requested a full tablet report, sending...
I20260809 00:46:28.395097 26441 ts_manager.cc:194] Registered new tserver with Master: 0c6d758a477b4a1a986364468a1f217f (127.30.104.193:46251)
I20260809 00:46:28.396214 26441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.193@KRBTEST.COM'} at 127.30.104.193:46723
WARNING: no policy specified for kudu/127.30.104.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
Entry for principal kudu/127.30.104.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab.
I20260809 00:46:28.418896 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44859
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:28.506439 26741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:28.506656 26741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:28.506681 26741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:28.507957 26741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:28.508003 26741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:28.508037 26741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:46:28.509593 26741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.194.keytab
--principal=kudu/127.30.104.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:28.509946 26741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:28.510752 26741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:28.517787 26751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:28.517761 26748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:28.517760 26746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:28.518025 26741 server_base.cc:1034] running on GCE node
I20260809 00:46:28.518399 26741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:28.518692 26741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:28.519886 26741 hybrid_clock.cc:648] HybridClock initialized: now 1786236388519875 us; error 41 us; skew 500 ppm
Aug 09 00:46:28 dist-test-slave-c12x krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:28.522835 26741 init.cc:377] Logged in from keytab as kudu/127.30.104.194@KRBTEST.COM (short username kudu)
I20260809 00:46:28.523331 26741 webserver.cc:460] Webserver started at http://127.30.104.194:36043/ using document root <none> and password file <none>
I20260809 00:46:28.523618 26741 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:28.523666 26741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:28.523761 26741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:28.524784 26741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "87e9b03ac62547f7a8dee67a09a4733f"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "660207c384453615481b2c8046f41bb4"
server_key_iv: "1c32bbd9b69487cc949d9add6014a402"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.525153 26741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "87e9b03ac62547f7a8dee67a09a4733f"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "660207c384453615481b2c8046f41bb4"
server_key_iv: "1c32bbd9b69487cc949d9add6014a402"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.526621 26741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:28.527408 26758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.527666 26741 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:28.527774 26741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "87e9b03ac62547f7a8dee67a09a4733f"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "660207c384453615481b2c8046f41bb4"
server_key_iv: "1c32bbd9b69487cc949d9add6014a402"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.527863 26741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:28.562953 26741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:28.565616 26741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:28.565804 26741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:28.566080 26741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:28.566447 26741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:28.566505 26741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.566542 26741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:28.566572 26741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.572779 26741 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:41317
I20260809 00:46:28.572829 26922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:41317 every 8 connection(s)
I20260809 00:46:28.573184 26741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:46:28.576043 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 26741
I20260809 00:46:28.576179 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:46:28.576378 31139 external_mini_cluster.cc:1428] Setting key 4c282de9ae6f1c3f623106aa6cde319e
Aug 09 00:46:28 dist-test-slave-c12x krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.194@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:28.584168 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.574047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:47871 (local address 127.30.104.254:44859)
0809 00:46:28.574113 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.574115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.574274 (+ 159us) server_negotiation.cc:407] Connection header received
0809 00:46:28.574767 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.574769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.574780 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.574855 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:28.575926 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.576760 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.577884 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.578204 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.578287 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.580980 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:28.580982 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:28.580982 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:28.580991 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:28.582663 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.583104 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.583105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.583106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.583177 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.583470 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.583470 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.583471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.583603 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:28.583683 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.583971 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.584072 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 00:46:28.584385 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.573934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44859 (local address 127.30.104.194:47871)
0809 00:46:28.574110 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.574155 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:28.574655 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:28.574911 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:28.574915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:28.575596 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:28.575807 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.575812 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.576889 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.576890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.577787 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.577790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.578362 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.578364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.578524 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.579138 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:28.579150 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:28.580853 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:28.582774 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:28.582781 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:28.582783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:28.583021 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:28.583249 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:28.583250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:28.583250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:28.583371 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:28.583674 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:28.583681 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:28.583834 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 00:46:28.583948 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 00:46:28.584120 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260809 00:46:28.584884 26923 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44859
I20260809 00:46:28.585011 26923 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:28.585233 26923 heartbeater.cc:511] Master 127.30.104.254:44859 requested a full tablet report, sending...
I20260809 00:46:28.585655 26441 ts_manager.cc:194] Registered new tserver with Master: 87e9b03ac62547f7a8dee67a09a4733f (127.30.104.194:41317)
I20260809 00:46:28.586048 26441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.194@KRBTEST.COM'} at 127.30.104.194:47871
WARNING: no policy specified for kudu/127.30.104.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
Entry for principal kudu/127.30.104.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab.
I20260809 00:46:28.605742 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44859
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:40297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:28.695622 26930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:28.695859 26930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:28.695888 26930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:28.697109 26930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:28.697165 26930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:28.697196 26930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:46:28.698830 26930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:40297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.195.keytab
--principal=kudu/127.30.104.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:44859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:28.699124 26930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:28.699404 26930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:28.705188 26937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:28.705215 26938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:28.705255 26943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:28.705194 26930 server_base.cc:1034] running on GCE node
I20260809 00:46:28.705736 26930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:28.706005 26930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:28.707207 26930 hybrid_clock.cc:648] HybridClock initialized: now 1786236388707189 us; error 37 us; skew 500 ppm
Aug 09 00:46:28 dist-test-slave-c12x krb5kdc[26372](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:28.710160 26930 init.cc:377] Logged in from keytab as kudu/127.30.104.195@KRBTEST.COM (short username kudu)
I20260809 00:46:28.710645 26930 webserver.cc:460] Webserver started at http://127.30.104.195:40193/ using document root <none> and password file <none>
I20260809 00:46:28.710868 26930 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:28.710912 26930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:28.711014 26930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:28.712294 26930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "a051fc1326a661b46f1517f2ef4561fc"
server_key_iv: "18ac1bae6aa653c9808bedad7646d50d"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.712633 26930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "a051fc1326a661b46f1517f2ef4561fc"
server_key_iv: "18ac1bae6aa653c9808bedad7646d50d"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.714116 26930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:46:28.714897 26951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.715106 26930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:46:28.715188 26930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8"
format_stamp: "Formatted at 2026-08-09 00:46:28 on dist-test-slave-c12x"
server_key: "a051fc1326a661b46f1517f2ef4561fc"
server_key_iv: "18ac1bae6aa653c9808bedad7646d50d"
server_key_version: "encryptionkey@0"
I20260809 00:46:28.715247 26930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:28.727281 26930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:28.729806 26930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:28.729928 26930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:28.730134 26930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:28.730489 26930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:28.730532 26930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.730582 26930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:28.730702 26930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:28.737140 26930 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:35721
I20260809 00:46:28.737188 27131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:35721 every 8 connection(s)
I20260809 00:46:28.737557 26930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:46:28.742198 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 26930
I20260809 00:46:28.742316 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:46:28.742563 31139 external_mini_cluster.cc:1428] Setting key 8a7bd6390c8c4b9e453f3dd8c56f4bd6
Aug 09 00:46:28 dist-test-slave-c12x krb5kdc[26372](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236388, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.195@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:28.748262 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.738376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:54433 (local address 127.30.104.254:44859)
0809 00:46:28.738430 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.738433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.738649 (+ 216us) server_negotiation.cc:407] Connection header received
0809 00:46:28.739172 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.739173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.739185 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.739254 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:28.740375 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.740883 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.742044 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.742360 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.742437 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.745114 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:28.745116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:28.745116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:28.745125 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:28.746674 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.747112 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.747113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.747114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.747177 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.747529 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.747531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.747531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.747674 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:28.747753 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.748110 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.748192 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:46:28.748564 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.738324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:44859 (local address 127.30.104.195:54433)
0809 00:46:28.738510 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.738555 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:28.739057 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:28.739336 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:28.739340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:28.740009 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:28.740239 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.740244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.740973 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.740974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.741923 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.741926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.742458 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.742459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.742568 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.743257 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:28.743268 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:28.745016 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:28.746771 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:28.746778 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:28.746780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:28.747017 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:28.747252 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:28.747253 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:28.747253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:28.747367 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:28.747778 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:28.747784 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:28.747925 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 00:46:28.748092 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 00:46:28.748277 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260809 00:46:28.749301 27132 heartbeater.cc:347] Connected to a master server at 127.30.104.254:44859
I20260809 00:46:28.749394 27132 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:28.749550 27132 heartbeater.cc:511] Master 127.30.104.254:44859 requested a full tablet report, sending...
I20260809 00:46:28.749971 26441 ts_manager.cc:194] Registered new tserver with Master: c29e2f7252e246ebb4ed82ec78ea7ba8 (127.30.104.195:35721)
I20260809 00:46:28.750398 26441 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.104.195@KRBTEST.COM'} at 127.30.104.195:54433
I20260809 00:46:28.754907 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:46:28.763156 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.756465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49688 (local address 127.30.104.254:44859)
0809 00:46:28.756559 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.756560 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.756633 (+ 73us) server_negotiation.cc:407] Connection header received
0809 00:46:28.756684 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.756685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.756694 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.756747 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:28.757413 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.757910 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.758620 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.758837 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.758922 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.760600 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:28.760602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:28.760602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:28.760610 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:28.761919 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.762287 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.762288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.762288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.762353 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:28.762568 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:28.762569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:28.762569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:28.762736 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:28.762859 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.762958 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.763047 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:46:28.764894 26441 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:46:28.766816 26441 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:46:28.774085 27152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.769623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:35721 (local address 127.0.0.1:44746)
0809 00:46:28.770160 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.770173 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:28.770231 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:28.770536 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:28.770537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:28.770539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:28.770600 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.770602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.771607 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.771608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.773069 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.773071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.773766 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.773847 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.773862 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:46:28.773938 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 00:46:28.773994 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":115,"thread_start_us":48,"threads_started":1}
I20260809 00:46:28.774179 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.769728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35460 (local address 127.30.104.193:46251)
0809 00:46:28.770015 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.770021 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.770032 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:46:28.770073 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.770075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.770093 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.770200 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:28.770387 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.771570 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773154 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.773815 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773916 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.773965 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.774048 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.774090 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":86,"threads_started":1}
I20260809 00:46:28.774273 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.769976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.30.104.194:41317)
0809 00:46:28.770138 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.770142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.770615 (+ 473us) server_negotiation.cc:407] Connection header received
0809 00:46:28.770660 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.770663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.770679 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.770761 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:28.770888 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.771882 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773019 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.773762 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773841 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.773901 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.774142 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.774183 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260809 00:46:28.774641 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.770008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.30.104.195:35721)
0809 00:46:28.770214 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.770219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.770231 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:46:28.770421 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.770423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.770441 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.770536 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:28.770659 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.771495 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773172 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.773658 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773765 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.773826 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.774542 (+ 716us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.774580 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260809 00:46:28.774185 27147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.769619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:46251 (local address 127.0.0.1:35460)
0809 00:46:28.769802 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.769872 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:28.769947 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:28.770209 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:28.770212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:28.770215 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:28.770301 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.770303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.771666 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.771667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.773067 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.773071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.773926 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.773980 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.773982 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:46:28.774033 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:46:28.774066 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":81,"thread_start_us":124,"threads_started":1}
I20260809 00:46:28.774294 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.769855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:41317 (local address 127.0.0.1:50560)
0809 00:46:28.770270 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.770279 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:28.770335 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:28.770775 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:28.770776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:28.770778 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:28.770820 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.770821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.771982 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.771983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.772927 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.772929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.773937 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.773938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.774004 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.774009 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:46:28.774060 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 00:46:28.774101 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":75,"threads_started":1}
I20260809 00:46:28.775617 26635 tablet_service.cc:1467] Processing CreateTablet for tablet c5cfe56dd3524ddfb735ca0af5010ee7 (DEFAULT_TABLE table=test-table [id=2048a168c5cd46b7b03837cdf8ca72b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:28.775652 26827 tablet_service.cc:1467] Processing CreateTablet for tablet c5cfe56dd3524ddfb735ca0af5010ee7 (DEFAULT_TABLE table=test-table [id=2048a168c5cd46b7b03837cdf8ca72b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:28.775861 27039 tablet_service.cc:1467] Processing CreateTablet for tablet c5cfe56dd3524ddfb735ca0af5010ee7 (DEFAULT_TABLE table=test-table [id=2048a168c5cd46b7b03837cdf8ca72b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:28.776022 26635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5cfe56dd3524ddfb735ca0af5010ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:28.776026 26827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5cfe56dd3524ddfb735ca0af5010ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:28.776193 27039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5cfe56dd3524ddfb735ca0af5010ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:28.778790 27156 tablet_bootstrap.cc:492] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f: Bootstrap starting.
I20260809 00:46:28.779588 27156 tablet_bootstrap.cc:654] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:28.779621 27160 tablet_bootstrap.cc:492] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: Bootstrap starting.
I20260809 00:46:28.780288 27156 log.cc:824] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:28.780388 27160 tablet_bootstrap.cc:654] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:28.780972 27160 log.cc:824] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:28.781196 27156 tablet_bootstrap.cc:492] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f: No bootstrap required, opened a new log
I20260809 00:46:28.781275 27156 ts_tablet_manager.cc:1397] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 00:46:28.781323 27158 tablet_bootstrap.cc:492] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Bootstrap starting.
I20260809 00:46:28.781746 27160 tablet_bootstrap.cc:492] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: No bootstrap required, opened a new log
I20260809 00:46:28.781834 27160 ts_tablet_manager.cc:1397] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:46:28.781960 27158 tablet_bootstrap.cc:654] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:28.782523 27158 log.cc:824] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:28.783313 27158 tablet_bootstrap.cc:492] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: No bootstrap required, opened a new log
I20260809 00:46:28.783476 27158 ts_tablet_manager.cc:1397] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:46:28.783480 27156 raft_consensus.cc:348] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.783675 27156 raft_consensus.cc:374] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:28.783747 27156 raft_consensus.cc:729] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6d758a477b4a1a986364468a1f217f, State: Initialized, Role: FOLLOWER
I20260809 00:46:28.783913 27156 consensus_queue.cc:260] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.783851 27160 raft_consensus.cc:348] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.784034 27160 raft_consensus.cc:374] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:28.784099 27160 raft_consensus.cc:729] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c29e2f7252e246ebb4ed82ec78ea7ba8, State: Initialized, Role: FOLLOWER
I20260809 00:46:28.784217 27156 ts_tablet_manager.cc:1428] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 00:46:28.784221 27160 consensus_queue.cc:260] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.784363 26731 heartbeater.cc:503] Master 127.30.104.254:44859 was elected leader, sending a full tablet report...
I20260809 00:46:28.784528 27160 ts_tablet_manager.cc:1428] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 00:46:28.784559 27132 heartbeater.cc:503] Master 127.30.104.254:44859 was elected leader, sending a full tablet report...
I20260809 00:46:28.785607 27158 raft_consensus.cc:348] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.785729 27158 raft_consensus.cc:374] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:28.785750 27158 raft_consensus.cc:729] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e9b03ac62547f7a8dee67a09a4733f, State: Initialized, Role: FOLLOWER
I20260809 00:46:28.785820 27158 consensus_queue.cc:260] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.786008 27158 ts_tablet_manager.cc:1428] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 00:46:28.786094 26923 heartbeater.cc:503] Master 127.30.104.254:44859 was elected leader, sending a full tablet report...
W20260809 00:46:28.823784 26925 tablet.cc:2369] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:46:28.855595 27170 raft_consensus.cc:482] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:28.855734 27170 raft_consensus.cc:504] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.856099 27170 leader_election.cc:288] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c6d758a477b4a1a986364468a1f217f (127.30.104.193:46251), c29e2f7252e246ebb4ed82ec78ea7ba8 (127.30.104.195:35721)
I20260809 00:46:28.859160 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.856536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:57493 (local address 127.30.104.193:46251)
0809 00:46:28.856584 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.856586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.856659 (+ 73us) server_negotiation.cc:407] Connection header received
0809 00:46:28.856722 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.856724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.856733 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.856790 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:28.856967 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.857464 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.858401 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.858780 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.858878 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.858887 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.859047 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.859085 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:46:28.859174 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.856489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:46237 (local address 127.30.104.195:35721)
0809 00:46:28.856551 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.856553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.856565 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:46:28.856615 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.856616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.856632 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.856752 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:28.856939 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.857429 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.858398 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.858788 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.858892 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.858902 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.859064 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.859101 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:46:28.859186 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.856347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:35721 (local address 127.30.104.194:46237)
0809 00:46:28.856403 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.856419 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:28.856487 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:28.856768 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:28.856770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:28.856772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:28.856854 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.856856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.857516 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.857517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.858292 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.858294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.858895 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.858895 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.858978 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.858980 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:46:28.859045 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 00:46:28.859085 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 00:46:28.859203 27171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.856351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:46251 (local address 127.30.104.194:57493)
0809 00:46:28.856606 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.856615 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:28.856668 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:28.856810 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:28.856810 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:28.856812 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:28.856888 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.856890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.857556 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.857557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.858294 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:28.858297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.858878 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.858879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:28.858951 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.858955 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:46:28.859025 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:46:28.859066 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":49,"thread_start_us":129,"threads_started":1}
I20260809 00:46:28.859609 27067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5cfe56dd3524ddfb735ca0af5010ee7" candidate_uuid: "87e9b03ac62547f7a8dee67a09a4733f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" is_pre_election: true
I20260809 00:46:28.859638 26670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5cfe56dd3524ddfb735ca0af5010ee7" candidate_uuid: "87e9b03ac62547f7a8dee67a09a4733f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6d758a477b4a1a986364468a1f217f" is_pre_election: true
I20260809 00:46:28.859782 27067 raft_consensus.cc:2454] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e9b03ac62547f7a8dee67a09a4733f in term 0.
I20260809 00:46:28.859793 26670 raft_consensus.cc:2454] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e9b03ac62547f7a8dee67a09a4733f in term 0.
I20260809 00:46:28.860033 26762 leader_election.cc:302] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6d758a477b4a1a986364468a1f217f, 87e9b03ac62547f7a8dee67a09a4733f; no voters:
I20260809 00:46:28.860179 27170 raft_consensus.cc:2781] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:46:28.860239 27170 raft_consensus.cc:482] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:46:28.860270 27170 raft_consensus.cc:3037] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:28.860984 27170 raft_consensus.cc:504] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.861186 27170 leader_election.cc:288] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [CANDIDATE]: Term 1 election: Requested vote from peers 0c6d758a477b4a1a986364468a1f217f (127.30.104.193:46251), c29e2f7252e246ebb4ed82ec78ea7ba8 (127.30.104.195:35721)
I20260809 00:46:28.861358 26670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5cfe56dd3524ddfb735ca0af5010ee7" candidate_uuid: "87e9b03ac62547f7a8dee67a09a4733f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6d758a477b4a1a986364468a1f217f"
I20260809 00:46:28.861485 27067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5cfe56dd3524ddfb735ca0af5010ee7" candidate_uuid: "87e9b03ac62547f7a8dee67a09a4733f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8"
I20260809 00:46:28.861459 26670 raft_consensus.cc:3037] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:28.861570 27067 raft_consensus.cc:3037] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:28.862243 26670 raft_consensus.cc:2454] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e9b03ac62547f7a8dee67a09a4733f in term 1.
I20260809 00:46:28.862382 27067 raft_consensus.cc:2454] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e9b03ac62547f7a8dee67a09a4733f in term 1.
I20260809 00:46:28.862421 26762 leader_election.cc:302] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6d758a477b4a1a986364468a1f217f, 87e9b03ac62547f7a8dee67a09a4733f; no voters:
I20260809 00:46:28.862501 27170 raft_consensus.cc:2781] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:28.862684 27170 raft_consensus.cc:686] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 1 LEADER]: Becoming Leader. State: Replica: 87e9b03ac62547f7a8dee67a09a4733f, State: Running, Role: LEADER
I20260809 00:46:28.862805 27170 consensus_queue.cc:237] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } }
I20260809 00:46:28.863782 26441 catalog_manager.cc:5495] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f reported cstate change: term changed from 0 to 1, leader changed from <none> to 87e9b03ac62547f7a8dee67a09a4733f (127.30.104.194). New cstate: current_term: 1 leader_uuid: "87e9b03ac62547f7a8dee67a09a4733f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e9b03ac62547f7a8dee67a09a4733f" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 41317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 } health_report { overall_health: UNKNOWN } } }
W20260809 00:46:28.882426 26733 tablet.cc:2369] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:46:28.891911 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.889547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50572 (local address 127.30.104.194:41317)
0809 00:46:28.889609 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:28.889611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:28.889620 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:46:28.889661 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:28.889662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:28.889690 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:28.889753 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:46:28.889889 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.890376 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.891103 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:28.891293 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:28.891348 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:28.891546 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:46:28.891651 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:46:28.891717 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:28.891755 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 00:46:28.891772 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:46:28.893862 27067 raft_consensus.cc:1261] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 1 FOLLOWER]: Refusing update from remote peer 87e9b03ac62547f7a8dee67a09a4733f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:28.893862 26670 raft_consensus.cc:1261] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 1 FOLLOWER]: Refusing update from remote peer 87e9b03ac62547f7a8dee67a09a4733f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:28.894306 27170 consensus_queue.cc:1035] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6d758a477b4a1a986364468a1f217f" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 46251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:28.894500 27172 consensus_queue.cc:1035] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c29e2f7252e246ebb4ed82ec78ea7ba8" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 35721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:28.899061 26440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:28.899197 26440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:28.900152 26440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4b17081dc1b741b3bc05cf042261a028: Sending DeleteTablet for 3 replicas of tablet c5cfe56dd3524ddfb735ca0af5010ee7
I20260809 00:46:28.900506 26635 tablet_service.cc:1514] Processing DeleteTablet for tablet c5cfe56dd3524ddfb735ca0af5010ee7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:28 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:35460
I20260809 00:46:28.900563 26827 tablet_service.cc:1514] Processing DeleteTablet for tablet c5cfe56dd3524ddfb735ca0af5010ee7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:28 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:50560
I20260809 00:46:28.900763 27196 tablet_replica.cc:330] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f: stopping tablet replica
I20260809 00:46:28.900816 27039 tablet_service.cc:1514] Processing DeleteTablet for tablet c5cfe56dd3524ddfb735ca0af5010ee7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:28 UTC) from {username='kudu', principal='kudu/127.30.104.254@KRBTEST.COM'} at 127.0.0.1:44746
I20260809 00:46:28.900883 27196 raft_consensus.cc:2229] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:28.901021 27196 raft_consensus.cc:2258] T c5cfe56dd3524ddfb735ca0af5010ee7 P 0c6d758a477b4a1a986364468a1f217f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:28.901059 27199 tablet_replica.cc:330] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: stopping tablet replica
I20260809 00:46:28.901218 27199 raft_consensus.cc:2229] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:28.901337 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 26540
I20260809 00:46:28.901356 27199 raft_consensus.cc:2258] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:28.906863 27199 ts_tablet_manager.cc:1905] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:28.909451 27199 ts_tablet_manager.cc:1918] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:28.909559 27199 log.cc:1196] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/c5cfe56dd3524ddfb735ca0af5010ee7
I20260809 00:46:28.909888 27199 ts_tablet_manager.cc:1939] T c5cfe56dd3524ddfb735ca0af5010ee7 P c29e2f7252e246ebb4ed82ec78ea7ba8: Deleting consensus metadata
I20260809 00:46:28.910974 26416 catalog_manager.cc:4841] TS c29e2f7252e246ebb4ed82ec78ea7ba8 (127.30.104.195:35721): tablet c5cfe56dd3524ddfb735ca0af5010ee7 (table test-table [id=2048a168c5cd46b7b03837cdf8ca72b0]) successfully deleted
I20260809 00:46:28.919643 27197 tablet_replica.cc:330] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: stopping tablet replica
I20260809 00:46:28.919842 27197 raft_consensus.cc:2229] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:46:28.920018 27197 raft_consensus.cc:2258] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:28.921106 27197 ts_tablet_manager.cc:1905] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:28.922703 27197 ts_tablet_manager.cc:1918] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:28.922807 27197 log.cc:1196] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/c5cfe56dd3524ddfb735ca0af5010ee7
I20260809 00:46:28.923127 27197 ts_tablet_manager.cc:1939] T c5cfe56dd3524ddfb735ca0af5010ee7 P 87e9b03ac62547f7a8dee67a09a4733f: Deleting consensus metadata
I20260809 00:46:28.923681 26418 catalog_manager.cc:4841] TS 87e9b03ac62547f7a8dee67a09a4733f (127.30.104.194:41317): tablet c5cfe56dd3524ddfb735ca0af5010ee7 (table test-table [id=2048a168c5cd46b7b03837cdf8ca72b0]) successfully deleted
W20260809 00:46:28.926981 26418 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:46251 (error 108)
I20260809 00:46:28.927268 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 26741
I20260809 00:46:28.931108 27152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:28.930919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:46251 (local address 127.0.0.1:35466)
0809 00:46:28.931005 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:28.931069 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:46251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 00:46:28.931288 26418 catalog_manager.cc:4584] TS 0c6d758a477b4a1a986364468a1f217f (127.30.104.193:46251): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c5cfe56dd3524ddfb735ca0af5010ee7: Network error: Client connection negotiation failed: client connection to 127.30.104.193:46251: connect: Connection refused (error 111)
I20260809 00:46:28.938880 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 26930
I20260809 00:46:28.956676 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 26388
2026-08-09T00:46:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2286 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T00:46:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:28Z Disabled control of system clock
I20260809 00:46:28.990799 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33011
--webserver_interface=127.30.104.254
--webserver_port=0
--builtin_ntp_servers=127.30.104.212:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:33011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:46:29.075945 27205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:29.076160 27205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:29.076181 27205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:29.077271 27205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:29.077308 27205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:29.077322 27205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:29.077333 27205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:29.077346 27205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:29.078642 27205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:33011
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:29.078838 27205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:29.079084 27205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:29.084039 27214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:29.084055 27217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:29.084076 27215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:29.084438 27205 server_base.cc:1034] running on GCE node
I20260809 00:46:29.084738 27205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:29.085034 27205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:29.086200 27205 hybrid_clock.cc:648] HybridClock initialized: now 1786236389086171 us; error 44 us; skew 500 ppm
I20260809 00:46:29.087560 27205 webserver.cc:460] Webserver started at http://127.30.104.254:41099/ using document root <none> and password file <none>
I20260809 00:46:29.087790 27205 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:29.087833 27205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:29.087929 27205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:29.088963 27205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "a70dbc3e77a34f6095e11b8fe74aed5b"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "fe7940cddddd95d6c858d6aa2c8916cd"
server_key_iv: "58240c70f7d33fe1ed6f3101ce4351c7"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.089278 27205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "a70dbc3e77a34f6095e11b8fe74aed5b"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "fe7940cddddd95d6c858d6aa2c8916cd"
server_key_iv: "58240c70f7d33fe1ed6f3101ce4351c7"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.090598 27205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:46:29.091401 27224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.091596 27205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:29.091693 27205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a70dbc3e77a34f6095e11b8fe74aed5b"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "fe7940cddddd95d6c858d6aa2c8916cd"
server_key_iv: "58240c70f7d33fe1ed6f3101ce4351c7"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.091753 27205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:29.130081 27205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:29.130411 27205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:29.130519 27205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:29.134488 27205 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:33011
I20260809 00:46:29.134582 27297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:33011 every 8 connection(s)
I20260809 00:46:29.134948 27205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:29.136101 27299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:29.136147 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 27205
I20260809 00:46:29.136267 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:29.136482 31139 external_mini_cluster.cc:1428] Setting key d4536ae7f7f7bffce272fc8006a33ce7
I20260809 00:46:29.139840 27299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b: Bootstrap starting.
I20260809 00:46:29.140470 27299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:29.140790 27299 log.cc:824] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:29.141535 27299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b: No bootstrap required, opened a new log
I20260809 00:46:29.142590 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.137071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49884 (local address 127.30.104.254:33011)
0809 00:46:29.137306 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.137316 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.137350 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:46:29.137885 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.137895 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.137983 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.138141 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:29.138332 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.139742 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.140595 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.141065 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.141222 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.141941 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:29.141945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:29.141945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:29.142110 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:29.142168 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:29.142179 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:29.142258 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.142335 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.142418 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260809 00:46:29.143741 27299 raft_consensus.cc:348] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dbc3e77a34f6095e11b8fe74aed5b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33011 } }
I20260809 00:46:29.143908 27299 raft_consensus.cc:374] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:29.143965 27299 raft_consensus.cc:729] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a70dbc3e77a34f6095e11b8fe74aed5b, State: Initialized, Role: FOLLOWER
I20260809 00:46:29.144091 27299 consensus_queue.cc:260] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dbc3e77a34f6095e11b8fe74aed5b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33011 } }
I20260809 00:46:29.144179 27299 raft_consensus.cc:388] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:29.144223 27299 raft_consensus.cc:482] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:29.144272 27299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:29.144817 27299 raft_consensus.cc:504] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dbc3e77a34f6095e11b8fe74aed5b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33011 } }
I20260809 00:46:29.144949 27299 leader_election.cc:302] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a70dbc3e77a34f6095e11b8fe74aed5b; no voters:
I20260809 00:46:29.145160 27299 leader_election.cc:288] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:29.145215 27306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:29.145385 27306 raft_consensus.cc:686] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [term 1 LEADER]: Becoming Leader. State: Replica: a70dbc3e77a34f6095e11b8fe74aed5b, State: Running, Role: LEADER
I20260809 00:46:29.145439 27299 sys_catalog.cc:564] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:29.145504 27306 consensus_queue.cc:237] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dbc3e77a34f6095e11b8fe74aed5b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33011 } }
I20260809 00:46:29.145927 27310 sys_catalog.cc:455] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a70dbc3e77a34f6095e11b8fe74aed5b. Latest consensus state: current_term: 1 leader_uuid: "a70dbc3e77a34f6095e11b8fe74aed5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dbc3e77a34f6095e11b8fe74aed5b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33011 } } }
I20260809 00:46:29.145920 27309 sys_catalog.cc:455] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a70dbc3e77a34f6095e11b8fe74aed5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a70dbc3e77a34f6095e11b8fe74aed5b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33011 } } }
I20260809 00:46:29.146031 27310 sys_catalog.cc:458] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:29.146039 27309 sys_catalog.cc:458] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:29.146462 27320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:29.147148 27320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:29.148789 27320 catalog_manager.cc:1347] Generated new cluster ID: 65761dbfb5fa48feba6607892d5df879
I20260809 00:46:29.148844 27320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:29.177615 27320 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:29.178206 27320 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:29.183362 27320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b: Generated new TSK 0
I20260809 00:46:29.183642 27320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:29.193593 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.193:0
--local_ip_for_outbound_sockets=127.30.104.193
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33011
--builtin_ntp_servers=127.30.104.212:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 00:46:29.286324 27337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:29.286552 27337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:29.286585 27337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:29.287936 27337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:29.287983 27337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:29.288012 27337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.193
I20260809 00:46:29.289491 27337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.104.193
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.193
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:29.289788 27337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:29.290179 27337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:29.296777 27351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:29.296798 27349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:29.296787 27347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:29.297258 27337 server_base.cc:1034] running on GCE node
I20260809 00:46:29.297544 27337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:29.297830 27337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:29.299122 27337 hybrid_clock.cc:648] HybridClock initialized: now 1786236389299099 us; error 36 us; skew 500 ppm
I20260809 00:46:29.301321 27337 webserver.cc:460] Webserver started at http://127.30.104.193:38917/ using document root <none> and password file <none>
I20260809 00:46:29.301766 27337 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:29.301864 27337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:29.302066 27337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:29.303642 27337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/data/instance:
uuid: "18af32bb9a564b919ee44d8a2978c428"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "95b5ccd28c8e1b3aa2cae95a2f9f22f8"
server_key_iv: "292fcc00f168b7e61edf6932624c67b9"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.304069 27337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance:
uuid: "18af32bb9a564b919ee44d8a2978c428"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "95b5ccd28c8e1b3aa2cae95a2f9f22f8"
server_key_iv: "292fcc00f168b7e61edf6932624c67b9"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.305619 27337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:46:29.306473 27359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.306707 27337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:29.306823 27337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/wal
uuid: "18af32bb9a564b919ee44d8a2978c428"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "95b5ccd28c8e1b3aa2cae95a2f9f22f8"
server_key_iv: "292fcc00f168b7e61edf6932624c67b9"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.306918 27337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:29.316622 27337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:29.316998 27337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:29.317165 27337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:29.317422 27337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:29.317798 27337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:29.317857 27337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.317902 27337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:29.317932 27337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.324322 27337 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.193:40977
I20260809 00:46:29.324371 27523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.193:40977 every 8 connection(s)
I20260809 00:46:29.324774 27337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/data/info.pb
I20260809 00:46:29.330577 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.325645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:55853 (local address 127.30.104.254:33011)
0809 00:46:29.325725 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.325728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.325956 (+ 228us) server_negotiation.cc:407] Connection header received
0809 00:46:29.326547 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.326549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.326569 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.326625 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:29.327374 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.327970 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.329041 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.329257 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.329341 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.330248 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:29.330250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:29.330250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:29.330256 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:29.330280 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:29.330284 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:29.330350 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.330451 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.330485 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 00:46:29.330863 27526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.325617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33011 (local address 127.30.104.193:55853)
0809 00:46:29.325820 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.325858 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:29.326412 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:29.326698 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:29.326703 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:29.326714 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:29.327253 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.327257 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.328088 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.328089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.328929 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.328931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.329352 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.329353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.329496 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.330091 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:29.330092 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:29.330123 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:29.330148 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:29.330352 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:29.330359 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:29.330362 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:29.330452 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 00:46:29.330601 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260809 00:46:29.331069 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 27337
I20260809 00:46:29.331202 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-0/wal/instance
I20260809 00:46:29.331432 31139 external_mini_cluster.cc:1428] Setting key bf9fe6f8a6a4311088e0c37005b508d2
I20260809 00:46:29.331470 27524 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33011
I20260809 00:46:29.331573 27524 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:29.331807 27524 heartbeater.cc:511] Master 127.30.104.254:33011 requested a full tablet report, sending...
I20260809 00:46:29.332409 27241 ts_manager.cc:194] Registered new tserver with Master: 18af32bb9a564b919ee44d8a2978c428 (127.30.104.193:40977)
I20260809 00:46:29.332618 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.194:0
--local_ip_for_outbound_sockets=127.30.104.194
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33011
--builtin_ntp_servers=127.30.104.212:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:46:29.333194 27241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.193:55853
W20260809 00:46:29.424117 27527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:29.424366 27527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:29.424391 27527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:29.425606 27527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:29.425650 27527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:29.425678 27527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.194
I20260809 00:46:29.427115 27527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.104.194
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.194
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:29.427361 27527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:29.427784 27527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:29.433735 27540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:29.433681 27536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:29.433694 27537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:29.433990 27527 server_base.cc:1034] running on GCE node
I20260809 00:46:29.434286 27527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:29.434583 27527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:29.435751 27527 hybrid_clock.cc:648] HybridClock initialized: now 1786236389435728 us; error 46 us; skew 500 ppm
I20260809 00:46:29.437408 27527 webserver.cc:460] Webserver started at http://127.30.104.194:43663/ using document root <none> and password file <none>
I20260809 00:46:29.437726 27527 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:29.437808 27527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:29.437970 27527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:29.439147 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/data/instance:
uuid: "7f162515f6b24377a8349d06c61a63a6"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "bcce6d40cfd8d1ad759c276650a38bec"
server_key_iv: "58aea39784e1eb3eb1bf5bb605fedbe3"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.439600 27527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance:
uuid: "7f162515f6b24377a8349d06c61a63a6"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "bcce6d40cfd8d1ad759c276650a38bec"
server_key_iv: "58aea39784e1eb3eb1bf5bb605fedbe3"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.441203 27527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:29.442277 27547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.442579 27527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:29.442718 27527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/wal
uuid: "7f162515f6b24377a8349d06c61a63a6"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "bcce6d40cfd8d1ad759c276650a38bec"
server_key_iv: "58aea39784e1eb3eb1bf5bb605fedbe3"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.442828 27527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:29.454659 27527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:29.455060 27527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:29.455242 27527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:29.455612 27527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:29.456028 27527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:29.456094 27527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.456135 27527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:29.456164 27527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.462725 27527 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.194:33551
I20260809 00:46:29.462817 27719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.194:33551 every 8 connection(s)
I20260809 00:46:29.463219 27527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/data/info.pb
I20260809 00:46:29.469285 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.464166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.194:58189 (local address 127.30.104.254:33011)
0809 00:46:29.464255 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.464258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.464683 (+ 425us) server_negotiation.cc:407] Connection header received
0809 00:46:29.465207 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.465209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.465221 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.465291 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:29.466055 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.466569 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.467750 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.467948 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.468016 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.469006 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:29.469007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:29.469007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:29.469014 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:29.469032 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:29.469037 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:29.469085 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.469190 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.469217 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 00:46:29.469559 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.464113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33011 (local address 127.30.104.194:58189)
0809 00:46:29.464376 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.464491 (+ 115us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:29.465095 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:29.465377 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:29.465381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:29.465389 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:29.465957 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.465961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.466709 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.466710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.467632 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.467635 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.468070 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.468071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.468202 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.468842 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:29.468842 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:29.468875 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:29.468900 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:29.469111 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:29.469116 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:29.469118 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 00:46:29.469174 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:46:29.469267 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":117,"threads_started":1}
I20260809 00:46:29.469887 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 27527
I20260809 00:46:29.470011 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/wal/instance
I20260809 00:46:29.470180 27721 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33011
I20260809 00:46:29.470237 31139 external_mini_cluster.cc:1428] Setting key 96e4476ae5f2fb875fb60d4c7a89a1c6
I20260809 00:46:29.470285 27721 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:29.470510 27721 heartbeater.cc:511] Master 127.30.104.254:33011 requested a full tablet report, sending...
I20260809 00:46:29.471019 27241 ts_manager.cc:194] Registered new tserver with Master: 7f162515f6b24377a8349d06c61a63a6 (127.30.104.194:33551)
I20260809 00:46:29.471355 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.104.195:0
--local_ip_for_outbound_sockets=127.30.104.195
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33011
--builtin_ntp_servers=127.30.104.212:46383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 00:46:29.471499 27241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.194:58189
W20260809 00:46:29.563134 27727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:29.563349 27727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:29.563370 27727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:29.564625 27727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:29.564673 27727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:29.564699 27727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.104.195
I20260809 00:46:29.566110 27727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:46383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.104.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.104.195
--webserver_port=0
--tserver_master_addrs=127.30.104.254:33011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.104.195
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:29.566373 27727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:29.566696 27727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:29.572451 27741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:29.572484 27727 server_base.cc:1034] running on GCE node
W20260809 00:46:29.572456 27737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:29.572453 27738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:29.573042 27727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:29.573341 27727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:29.574497 27727 hybrid_clock.cc:648] HybridClock initialized: now 1786236389574476 us; error 39 us; skew 500 ppm
I20260809 00:46:29.575934 27727 webserver.cc:460] Webserver started at http://127.30.104.195:35343/ using document root <none> and password file <none>
I20260809 00:46:29.576211 27727 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:29.576283 27727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:29.576475 27727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:29.577626 27727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/data/instance:
uuid: "e488f506cec6433693fa1a55723a1e31"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "fc3e2ec0107d2ac0a67dcd334b3f2dc5"
server_key_iv: "02b26d9f993fb4e0dd11b4c8c9e12b33"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.578018 27727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance:
uuid: "e488f506cec6433693fa1a55723a1e31"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "fc3e2ec0107d2ac0a67dcd334b3f2dc5"
server_key_iv: "02b26d9f993fb4e0dd11b4c8c9e12b33"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.579552 27727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:29.580416 27748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.580667 27727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:29.580777 27727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/wal
uuid: "e488f506cec6433693fa1a55723a1e31"
format_stamp: "Formatted at 2026-08-09 00:46:29 on dist-test-slave-c12x"
server_key: "fc3e2ec0107d2ac0a67dcd334b3f2dc5"
server_key_iv: "02b26d9f993fb4e0dd11b4c8c9e12b33"
server_key_version: "encryptionkey@0"
I20260809 00:46:29.580847 27727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:29.593490 27727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:29.593788 27727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:29.593895 27727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:29.594115 27727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 00:46:29.594533 27727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 00:46:29.594573 27727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.594596 27727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 00:46:29.594642 27727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:29.600885 27727 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.195:44759
I20260809 00:46:29.600929 27907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.195:44759 every 8 connection(s)
I20260809 00:46:29.601300 27727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/data/info.pb
I20260809 00:46:29.606912 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.602094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.195:53223 (local address 127.30.104.254:33011)
0809 00:46:29.602168 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.602170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.602376 (+ 206us) server_negotiation.cc:407] Connection header received
0809 00:46:29.602956 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.602958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.602971 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.603021 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:29.603834 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.604342 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.605378 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.605588 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.605670 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.606604 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:29.606605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:29.606606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:29.606611 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:29.606633 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:29.606637 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:29.606698 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.606796 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.606823 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 00:46:29.607191 27911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.602039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33011 (local address 127.30.104.195:53223)
0809 00:46:29.602248 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.602284 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:29.602837 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:29.603093 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:29.603098 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:29.603109 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:29.603736 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.603742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.604449 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.604450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.605276 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.605279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.605678 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.605680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.605821 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.606456 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 00:46:29.606457 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:29.606486 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 00:46:29.606511 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:29.606699 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:29.606706 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:29.606709 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 00:46:29.606798 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 00:46:29.606909 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":74,"threads_started":1}
I20260809 00:46:29.607750 27908 heartbeater.cc:347] Connected to a master server at 127.30.104.254:33011
I20260809 00:46:29.607898 27908 heartbeater.cc:464] Registering TS with master...
I20260809 00:46:29.608068 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 27727
I20260809 00:46:29.608180 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/wal/instance
I20260809 00:46:29.608136 27908 heartbeater.cc:511] Master 127.30.104.254:33011 requested a full tablet report, sending...
I20260809 00:46:29.608390 31139 external_mini_cluster.cc:1428] Setting key d61404ea3a5700ea8c57e719611507ef
I20260809 00:46:29.608608 27241 ts_manager.cc:194] Registered new tserver with Master: e488f506cec6433693fa1a55723a1e31 (127.30.104.195:44759)
I20260809 00:46:29.609122 27241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.104.195:53223
I20260809 00:46:29.609400 31139 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 00:46:29.613646 27304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.610886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49892 (local address 127.30.104.254:33011)
0809 00:46:29.610974 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.610975 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.611052 (+ 77us) server_negotiation.cc:407] Connection header received
0809 00:46:29.611108 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.611108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.611117 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.611177 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:29.611310 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.611859 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.612600 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.612781 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.612833 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.613467 (+ 634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:29.613468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:29.613469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:29.613473 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:29.613485 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:29.613488 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:29.613531 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.613565 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.613585 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 00:46:29.615696 27240 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 00:46:29.617341 27240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 00:46:29.625273 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.620120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:44759 (local address 127.0.0.1:57814)
0809 00:46:29.620982 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.620993 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:29.621049 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:29.621237 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:29.621238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:29.621240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:29.621296 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.621298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.622607 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.622608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.624032 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.624035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.625016 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.625017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.625087 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.625091 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:46:29.625140 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 00:46:29.625186 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":38,"thread_start_us":64,"threads_started":1}
I20260809 00:46:29.625273 27920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.620065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:33551 (local address 127.0.0.1:42394)
0809 00:46:29.620271 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.620318 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:29.620390 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:29.620901 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:29.620904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:29.620907 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:29.620985 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.620987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.622588 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.622589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.624032 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.624035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.624910 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.624911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.624997 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.625012 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 00:46:29.625076 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 00:46:29.625135 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":75,"thread_start_us":86,"threads_started":1}
I20260809 00:46:29.625567 27919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.620537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:40977 (local address 127.0.0.1:57382)
0809 00:46:29.621781 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.621793 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:29.621850 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:29.622116 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:29.622117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:29.622119 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:29.622164 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.622165 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.623397 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.623398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.624314 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.624317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.625316 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.625317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.625371 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.625376 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:46:29.625432 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 00:46:29.625484 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1230,"thread_start_us":75,"threads_started":1}
I20260809 00:46:29.625648 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.620341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57814 (local address 127.30.104.195:44759)
0809 00:46:29.620538 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.620543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.621063 (+ 520us) server_negotiation.cc:407] Connection header received
0809 00:46:29.621129 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.621132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.621149 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.621228 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:29.621379 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.622510 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.624127 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.624758 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.624848 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.624954 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.625502 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.625559 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260809 00:46:29.625825 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.620639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.30.104.193:40977)
0809 00:46:29.621138 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.621143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.621913 (+ 770us) server_negotiation.cc:407] Connection header received
0809 00:46:29.621959 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.621962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.621977 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.622053 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:29.622243 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.623314 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.624408 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.625131 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.625197 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.625280 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.625670 (+ 390us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.625719 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":154,"threads_started":1}
I20260809 00:46:29.626121 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.620270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42394 (local address 127.30.104.194:33551)
0809 00:46:29.620707 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.620712 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.620724 (+ 12us) server_negotiation.cc:407] Connection header received
0809 00:46:29.620768 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.620770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.620786 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.620879 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:29.621293 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.622472 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.624138 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.624805 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.625240 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.625322 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.626009 (+ 687us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.626057 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":99,"threads_started":1}
I20260809 00:46:29.626374 27817 tablet_service.cc:1467] Processing CreateTablet for tablet fe807cd7d09f4bef9d40bcd44948329b (DEFAULT_TABLE table=test-table [id=d398c2742f5646808c88923e708ae2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:29.626766 27817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe807cd7d09f4bef9d40bcd44948329b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:29.626767 27620 tablet_service.cc:1467] Processing CreateTablet for tablet fe807cd7d09f4bef9d40bcd44948329b (DEFAULT_TABLE table=test-table [id=d398c2742f5646808c88923e708ae2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:29.627123 27620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe807cd7d09f4bef9d40bcd44948329b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:29.628121 27428 tablet_service.cc:1467] Processing CreateTablet for tablet fe807cd7d09f4bef9d40bcd44948329b (DEFAULT_TABLE table=test-table [id=d398c2742f5646808c88923e708ae2f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 00:46:29.628525 27428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe807cd7d09f4bef9d40bcd44948329b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:29.630105 27928 tablet_bootstrap.cc:492] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Bootstrap starting.
I20260809 00:46:29.630553 27929 tablet_bootstrap.cc:492] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Bootstrap starting.
I20260809 00:46:29.630918 27928 tablet_bootstrap.cc:654] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:29.631223 27929 tablet_bootstrap.cc:654] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:29.631261 27928 log.cc:824] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:29.631580 27929 log.cc:824] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:29.632221 27932 tablet_bootstrap.cc:492] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428: Bootstrap starting.
I20260809 00:46:29.632385 27929 tablet_bootstrap.cc:492] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: No bootstrap required, opened a new log
I20260809 00:46:29.632457 27929 ts_tablet_manager.cc:1397] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:46:29.632846 27932 tablet_bootstrap.cc:654] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:29.633045 27928 tablet_bootstrap.cc:492] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: No bootstrap required, opened a new log
I20260809 00:46:29.633124 27928 ts_tablet_manager.cc:1397] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 00:46:29.633213 27932 log.cc:824] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:29.633948 27932 tablet_bootstrap.cc:492] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428: No bootstrap required, opened a new log
I20260809 00:46:29.634049 27932 ts_tablet_manager.cc:1397] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 00:46:29.635278 27929 raft_consensus.cc:348] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.635495 27929 raft_consensus.cc:374] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:29.635541 27929 raft_consensus.cc:729] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e488f506cec6433693fa1a55723a1e31, State: Initialized, Role: FOLLOWER
I20260809 00:46:29.635690 27929 consensus_queue.cc:260] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.635787 27928 raft_consensus.cc:348] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.635949 27928 raft_consensus.cc:374] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:29.635970 27929 ts_tablet_manager.cc:1428] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 00:46:29.635993 27928 raft_consensus.cc:729] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f162515f6b24377a8349d06c61a63a6, State: Initialized, Role: FOLLOWER
I20260809 00:46:29.636124 27908 heartbeater.cc:503] Master 127.30.104.254:33011 was elected leader, sending a full tablet report...
I20260809 00:46:29.636113 27928 consensus_queue.cc:260] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.636130 27932 raft_consensus.cc:348] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.636291 27932 raft_consensus.cc:374] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:29.636334 27932 raft_consensus.cc:729] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18af32bb9a564b919ee44d8a2978c428, State: Initialized, Role: FOLLOWER
I20260809 00:46:29.636391 27928 ts_tablet_manager.cc:1428] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Time spent starting tablet: real 0.003s user 0.002s sys 0.002s
I20260809 00:46:29.636451 27932 consensus_queue.cc:260] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.636556 27721 heartbeater.cc:503] Master 127.30.104.254:33011 was elected leader, sending a full tablet report...
I20260809 00:46:29.636734 27932 ts_tablet_manager.cc:1428] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 00:46:29.636765 27524 heartbeater.cc:503] Master 127.30.104.254:33011 was elected leader, sending a full tablet report...
W20260809 00:46:29.714004 27722 tablet.cc:2369] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:46:29.825618 27525 tablet.cc:2369] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 00:46:29.851894 27909 tablet.cc:2369] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 00:46:29.878511 27937 raft_consensus.cc:482] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 00:46:29.878721 27937 raft_consensus.cc:504] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.879191 27937 leader_election.cc:288] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e488f506cec6433693fa1a55723a1e31 (127.30.104.195:44759), 7f162515f6b24377a8349d06c61a63a6 (127.30.104.194:33551)
I20260809 00:46:29.882395 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.879395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.194:33551 (local address 127.30.104.193:60185)
0809 00:46:29.879584 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.879597 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:29.879672 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:29.879872 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:29.879873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:29.879875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:29.879959 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.879961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.880648 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.880649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.881465 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.881467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.882090 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.882091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.882165 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.882170 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 00:46:29.882236 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 00:46:29.882286 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":4,"thread_start_us":87,"threads_started":1}
I20260809 00:46:29.882445 27921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.879501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:60185 (local address 127.30.104.194:33551)
0809 00:46:29.879554 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.879557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.879677 (+ 120us) server_negotiation.cc:407] Connection header received
0809 00:46:29.879775 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.879776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.879787 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.879847 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:29.880052 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.880553 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.881563 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.881994 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.882081 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.882091 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.882303 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.882351 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 00:46:29.882651 27922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.879531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.104.193:37197 (local address 127.30.104.195:44759)
0809 00:46:29.879601 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.879603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.879726 (+ 123us) server_negotiation.cc:407] Connection header received
0809 00:46:29.879803 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.879804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.879814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.879876 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 00:46:29.880076 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.880553 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.881735 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.882217 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.882283 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.882290 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.882527 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.882578 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:46:29.882779 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe807cd7d09f4bef9d40bcd44948329b" candidate_uuid: "18af32bb9a564b919ee44d8a2978c428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f162515f6b24377a8349d06c61a63a6" is_pre_election: true
I20260809 00:46:29.882884 27938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.879433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.195:44759 (local address 127.30.104.193:37197)
0809 00:46:29.879614 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.879623 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:29.879740 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:29.879899 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:29.879900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:29.879902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 00:46:29.879981 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.879982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.880656 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.880658 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.881626 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:29.881628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.882321 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.882322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:29.882410 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.882414 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 00:46:29.882742 (+ 328us) client_negotiation.cc:240] Negotiation successful
0809 00:46:29.882798 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":38,"thread_start_us":81,"threads_started":1}
I20260809 00:46:29.882968 27656 raft_consensus.cc:2454] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18af32bb9a564b919ee44d8a2978c428 in term 0.
I20260809 00:46:29.883246 27841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe807cd7d09f4bef9d40bcd44948329b" candidate_uuid: "18af32bb9a564b919ee44d8a2978c428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e488f506cec6433693fa1a55723a1e31" is_pre_election: true
I20260809 00:46:29.883396 27841 raft_consensus.cc:2454] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 18af32bb9a564b919ee44d8a2978c428 in term 0.
I20260809 00:46:29.883316 27362 leader_election.cc:302] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18af32bb9a564b919ee44d8a2978c428, 7f162515f6b24377a8349d06c61a63a6; no voters:
I20260809 00:46:29.883544 27937 raft_consensus.cc:2781] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 00:46:29.883610 27937 raft_consensus.cc:482] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 00:46:29.883628 27937 raft_consensus.cc:3037] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:29.884394 27937 raft_consensus.cc:504] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.884557 27937 leader_election.cc:288] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [CANDIDATE]: Term 1 election: Requested vote from peers e488f506cec6433693fa1a55723a1e31 (127.30.104.195:44759), 7f162515f6b24377a8349d06c61a63a6 (127.30.104.194:33551)
I20260809 00:46:29.884704 27841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe807cd7d09f4bef9d40bcd44948329b" candidate_uuid: "18af32bb9a564b919ee44d8a2978c428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e488f506cec6433693fa1a55723a1e31"
I20260809 00:46:29.884732 27656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe807cd7d09f4bef9d40bcd44948329b" candidate_uuid: "18af32bb9a564b919ee44d8a2978c428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f162515f6b24377a8349d06c61a63a6"
I20260809 00:46:29.884793 27841 raft_consensus.cc:3037] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:29.884838 27656 raft_consensus.cc:3037] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:29.885578 27656 raft_consensus.cc:2454] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18af32bb9a564b919ee44d8a2978c428 in term 1.
I20260809 00:46:29.885584 27841 raft_consensus.cc:2454] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 18af32bb9a564b919ee44d8a2978c428 in term 1.
I20260809 00:46:29.885774 27363 leader_election.cc:302] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 18af32bb9a564b919ee44d8a2978c428, e488f506cec6433693fa1a55723a1e31; no voters:
I20260809 00:46:29.885893 27937 raft_consensus.cc:2781] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:29.886109 27937 raft_consensus.cc:686] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 1 LEADER]: Becoming Leader. State: Replica: 18af32bb9a564b919ee44d8a2978c428, State: Running, Role: LEADER
I20260809 00:46:29.886217 27937 consensus_queue.cc:237] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } }
I20260809 00:46:29.887161 27240 catalog_manager.cc:5495] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 reported cstate change: term changed from 0 to 1, leader changed from <none> to 18af32bb9a564b919ee44d8a2978c428 (127.30.104.193). New cstate: current_term: 1 leader_uuid: "18af32bb9a564b919ee44d8a2978c428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "18af32bb9a564b919ee44d8a2978c428" member_type: VOTER last_known_addr { host: "127.30.104.193" port: 40977 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 } health_report { overall_health: UNKNOWN } } }
I20260809 00:46:29.913374 27927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.910988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.30.104.193:40977)
0809 00:46:29.911053 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:29.911055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:29.911066 (+ 11us) server_negotiation.cc:407] Connection header received
0809 00:46:29.911145 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:29.911146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:29.911167 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:29.911236 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:46:29.911357 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.911870 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.912639 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:29.912820 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:29.912871 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:29.913018 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:46:29.913127 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:46:29.913191 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:29.913293 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:46:29.913312 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:46:29.915220 27841 raft_consensus.cc:1261] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 1 FOLLOWER]: Refusing update from remote peer 18af32bb9a564b919ee44d8a2978c428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:29.915304 27656 raft_consensus.cc:1261] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 1 FOLLOWER]: Refusing update from remote peer 18af32bb9a564b919ee44d8a2978c428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:29.915693 27937 consensus_queue.cc:1035] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f162515f6b24377a8349d06c61a63a6" member_type: VOTER last_known_addr { host: "127.30.104.194" port: 33551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:29.915859 27941 consensus_queue.cc:1035] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e488f506cec6433693fa1a55723a1e31" member_type: VOTER last_known_addr { host: "127.30.104.195" port: 44759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:29.919212 27945 mvcc.cc:205] Tried to move back new op lower bound from 7316424253090050048 to 7316424252974596096. Current Snapshot: MvccSnapshot[applied={T|T < 7316424253090050048}]
I20260809 00:46:29.919335 27947 mvcc.cc:205] Tried to move back new op lower bound from 7316424253090050048 to 7316424252974596096. Current Snapshot: MvccSnapshot[applied={T|T < 7316424253090050048 or (T in {7316424253090050048})}]
I20260809 00:46:29.921898 27240 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:29.922009 27240 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 00:46:29.922887 27240 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a70dbc3e77a34f6095e11b8fe74aed5b: Sending DeleteTablet for 3 replicas of tablet fe807cd7d09f4bef9d40bcd44948329b
I20260809 00:46:29.923295 27428 tablet_service.cc:1514] Processing DeleteTablet for tablet fe807cd7d09f4bef9d40bcd44948329b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:29 UTC) from {username='slave'} at 127.0.0.1:57382
I20260809 00:46:29.923311 27620 tablet_service.cc:1514] Processing DeleteTablet for tablet fe807cd7d09f4bef9d40bcd44948329b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:29 UTC) from {username='slave'} at 127.0.0.1:42394
I20260809 00:46:29.923520 27817 tablet_service.cc:1514] Processing DeleteTablet for tablet fe807cd7d09f4bef9d40bcd44948329b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 00:46:29 UTC) from {username='slave'} at 127.0.0.1:57814
I20260809 00:46:29.923606 27954 tablet_replica.cc:330] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: stopping tablet replica
I20260809 00:46:29.923595 27955 tablet_replica.cc:330] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428: stopping tablet replica
I20260809 00:46:29.923769 27955 raft_consensus.cc:2229] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 1 LEADER]: Raft consensus shutting down.
I20260809 00:46:29.923797 27954 raft_consensus.cc:2229] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:29.923919 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 27337
I20260809 00:46:29.923957 27955 raft_consensus.cc:2258] T fe807cd7d09f4bef9d40bcd44948329b P 18af32bb9a564b919ee44d8a2978c428 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:29.928571 27957 tablet_replica.cc:330] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: stopping tablet replica
I20260809 00:46:29.928715 27957 raft_consensus.cc:2229] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 00:46:29.928826 27957 raft_consensus.cc:2258] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:29.930727 27957 ts_tablet_manager.cc:1905] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:29.932493 27957 ts_tablet_manager.cc:1918] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:29.932607 27957 log.cc:1196] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-2/wal/wals/fe807cd7d09f4bef9d40bcd44948329b
I20260809 00:46:29.932940 27957 ts_tablet_manager.cc:1939] T fe807cd7d09f4bef9d40bcd44948329b P e488f506cec6433693fa1a55723a1e31: Deleting consensus metadata
I20260809 00:46:29.933472 27228 catalog_manager.cc:4841] TS e488f506cec6433693fa1a55723a1e31 (127.30.104.195:44759): tablet fe807cd7d09f4bef9d40bcd44948329b (table test-table [id=d398c2742f5646808c88923e708ae2f8]) successfully deleted
I20260809 00:46:29.934060 27954 raft_consensus.cc:2258] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 00:46:29.934623 27954 ts_tablet_manager.cc:1905] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 00:46:29.936003 27954 ts_tablet_manager.cc:1918] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 00:46:29.936112 27954 log.cc:1196] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Deleting WAL directory at /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786236131192310-31139-0/minicluster-data/ts-1/wal/wals/fe807cd7d09f4bef9d40bcd44948329b
I20260809 00:46:29.936365 27954 ts_tablet_manager.cc:1939] T fe807cd7d09f4bef9d40bcd44948329b P 7f162515f6b24377a8349d06c61a63a6: Deleting consensus metadata
I20260809 00:46:29.936838 27227 catalog_manager.cc:4841] TS 7f162515f6b24377a8349d06c61a63a6 (127.30.104.194:33551): tablet fe807cd7d09f4bef9d40bcd44948329b (table test-table [id=d398c2742f5646808c88923e708ae2f8]) successfully deleted
W20260809 00:46:29.938423 27226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.104.193:40977 (error 108)
I20260809 00:46:29.938602 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 27527
I20260809 00:46:29.956229 27924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:29.938902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.193:40977 (local address 127.0.0.1:57404)
0809 00:46:29.956049 (+ 17147us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:29.956161 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.193:40977: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17097}
W20260809 00:46:29.956477 27226 catalog_manager.cc:4584] TS 18af32bb9a564b919ee44d8a2978c428 (127.30.104.193:40977): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe807cd7d09f4bef9d40bcd44948329b: Network error: Client connection negotiation failed: client connection to 127.30.104.193:40977: connect: Connection refused (error 111)
I20260809 00:46:29.960628 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 27727
I20260809 00:46:29.975657 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 27205
2026-08-09T00:46:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1018 ms)
[----------] 6 tests from EncryptionPolicyTest (9709 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:30 dist-test-slave-c12x krb5kdc[27962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:30 dist-test-slave-c12x krb5kdc[27962](info): set up 2 sockets
Aug 09 00:46:30 dist-test-slave-c12x krb5kdc[27962](info): commencing operation
krb5kdc: starting...
W20260809 00:46:31.198366 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:31 dist-test-slave-c12x krb5kdc[27962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:31.336746 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:31.426419 27978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:31.426661 27978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:31.426688 27978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:31.427995 27978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:31.428059 27978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:31.428074 27978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:31.428087 27978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:31.428099 27978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:31.429848 27978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33019
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:31.430150 27978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:31.430400 27978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 00:46:31.435945 27978 server_base.cc:1034] running on GCE node
W20260809 00:46:31.435868 27986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:31.435901 27987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:31.435877 27991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:31.436539 27978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:31.436874 27978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:31.438089 27978 hybrid_clock.cc:648] HybridClock initialized: now 1786236391438071 us; error 45 us; skew 500 ppm
Aug 09 00:46:31 dist-test-slave-c12x krb5kdc[27962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236391, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:31.446229 27978 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:31.446703 27978 webserver.cc:460] Webserver started at http://127.0.0.1:40531/ using document root <none> and password file <none>
I20260809 00:46:31.446983 27978 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:31.447034 27978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:31.447186 27978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:31.448385 27978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "a816decfd2254768851f89c1fce3362a"
format_stamp: "Formatted at 2026-08-09 00:46:31 on dist-test-slave-c12x"
server_key: "032c4337d0a21367b7c32dfe3e2913aa"
server_key_iv: "e701fa96731e41fef9692ab5427d9414"
server_key_version: "encryptionkey@0"
I20260809 00:46:31.448793 27978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "a816decfd2254768851f89c1fce3362a"
format_stamp: "Formatted at 2026-08-09 00:46:31 on dist-test-slave-c12x"
server_key: "032c4337d0a21367b7c32dfe3e2913aa"
server_key_iv: "e701fa96731e41fef9692ab5427d9414"
server_key_version: "encryptionkey@0"
I20260809 00:46:31.450289 27978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:46:31.451154 28002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:31.451386 27978 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:31.451524 27978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a816decfd2254768851f89c1fce3362a"
format_stamp: "Formatted at 2026-08-09 00:46:31 on dist-test-slave-c12x"
server_key: "032c4337d0a21367b7c32dfe3e2913aa"
server_key_iv: "e701fa96731e41fef9692ab5427d9414"
server_key_version: "encryptionkey@0"
I20260809 00:46:31.451588 27978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:31.476781 27978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:31.479291 27978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:31.479411 27978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:31.483618 27978 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33019
I20260809 00:46:31.483667 28084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33019 every 8 connection(s)
I20260809 00:46:31.484063 27978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:31.485106 28086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:31.487555 28086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a: Bootstrap starting.
I20260809 00:46:31.488153 28086 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:31.488404 28086 log.cc:824] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:31.489202 28086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a: No bootstrap required, opened a new log
I20260809 00:46:31.490811 28086 raft_consensus.cc:348] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a816decfd2254768851f89c1fce3362a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33019 } }
I20260809 00:46:31.490970 28086 raft_consensus.cc:374] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:31.490994 28086 raft_consensus.cc:729] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a816decfd2254768851f89c1fce3362a, State: Initialized, Role: FOLLOWER
I20260809 00:46:31.491106 28086 consensus_queue.cc:260] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a816decfd2254768851f89c1fce3362a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33019 } }
I20260809 00:46:31.491204 28086 raft_consensus.cc:388] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:31.491243 28086 raft_consensus.cc:482] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:31.491304 28086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:31.491931 28086 raft_consensus.cc:504] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a816decfd2254768851f89c1fce3362a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33019 } }
I20260809 00:46:31.492063 28086 leader_election.cc:302] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a816decfd2254768851f89c1fce3362a; no voters:
I20260809 00:46:31.492275 28086 leader_election.cc:288] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:31.492333 28090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:31.492581 28086 sys_catalog.cc:564] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:31.492614 28090 raft_consensus.cc:686] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [term 1 LEADER]: Becoming Leader. State: Replica: a816decfd2254768851f89c1fce3362a, State: Running, Role: LEADER
I20260809 00:46:31.492826 28090 consensus_queue.cc:237] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a816decfd2254768851f89c1fce3362a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33019 } }
I20260809 00:46:31.493292 28093 sys_catalog.cc:455] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a816decfd2254768851f89c1fce3362a. Latest consensus state: current_term: 1 leader_uuid: "a816decfd2254768851f89c1fce3362a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a816decfd2254768851f89c1fce3362a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33019 } } }
I20260809 00:46:31.493448 28091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a816decfd2254768851f89c1fce3362a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a816decfd2254768851f89c1fce3362a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33019 } } }
I20260809 00:46:31.493564 28091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:31.493793 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 27978
I20260809 00:46:31.493952 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:31.494196 31139 external_mini_cluster.cc:1428] Setting key 2906691dfa88394d9de907d414033980
I20260809 00:46:31.493479 28093 sys_catalog.cc:458] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a [sys.catalog]: This master's current role is: LEADER
W20260809 00:46:31.494920 28110 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:46:31.494984 28110 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:46:31.495107 28101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:31.495304 28101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:31.497267 28101 catalog_manager.cc:1347] Generated new cluster ID: cc7a669764a0461f9ff5abbcaa2830d6
I20260809 00:46:31.497332 28101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:46:31 dist-test-slave-c12x krb5kdc[27962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:46:31.505726 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:31.494757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58745 (local address 127.0.0.1:33019)
0809 00:46:31.495110 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:31.495121 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:31.495152 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:46:31.495305 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:31.495316 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:31.495388 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:31.495552 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:31.496261 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:31.497427 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:31.498200 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:31.498648 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:31.498773 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:31.501552 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:31.501557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:31.501557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:31.501570 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:31.504033 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:31.504468 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:31.504472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:31.504472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:31.504578 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:31.504830 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:31.504831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:31.504832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:31.505102 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:31.505296 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:31.505387 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 00:46:31.505502 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":83,"threads_started":1}
I20260809 00:46:31.506860 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:31.508919 28101 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:31.509763 28101 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:31.515295 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:31.508401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37517 (local address 127.0.0.1:33019)
0809 00:46:31.508458 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:31.508460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:31.508591 (+ 131us) server_negotiation.cc:407] Connection header received
0809 00:46:31.508639 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:31.508640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:31.508651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:31.508730 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:31.509387 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:31.509988 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:31.510716 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:31.510955 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:31.511035 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:31.512353 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:31.512354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:31.512355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:31.512362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:31.513852 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:31.514246 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:31.514248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:31.514248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:31.514313 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:31.514552 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:31.514553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:31.514554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:31.514776 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:31.514940 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:31.515102 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 00:46:31.515175 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:46:31.523713 28101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a816decfd2254768851f89c1fce3362a: Generated new TSK 0
I20260809 00:46:31.523960 28101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:31.529683 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 27978
W20260809 00:46:31.542725 27959 connection.cc:535] client connection to 127.0.0.1:33019 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33019): Connection reset by peer (error 104)
2026-08-09T00:46:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1556 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:31 dist-test-slave-c12x krb5kdc[28130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:31 dist-test-slave-c12x krb5kdc[28130](info): set up 2 sockets
Aug 09 00:46:31 dist-test-slave-c12x krb5kdc[28130](info): commencing operation
krb5kdc: starting...
W20260809 00:46:32.778995 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:32 dist-test-slave-c12x krb5kdc[28130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:32.914338 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39473
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:32.998832 28150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:32.999135 28150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:32.999177 28150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:33.000447 28150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:33.000535 28150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:33.000560 28150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:33.000581 28150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:33.000612 28150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:33.002209 28150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39473
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:33.002530 28150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:33.002784 28150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:33.008189 28158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:33.008213 28150 server_base.cc:1034] running on GCE node
W20260809 00:46:33.008213 28159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:33.008193 28161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:33.008665 28150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:33.009037 28150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:33.010237 28150 hybrid_clock.cc:648] HybridClock initialized: now 1786236393010216 us; error 42 us; skew 500 ppm
Aug 09 00:46:33 dist-test-slave-c12x krb5kdc[28130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236393, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:33.013052 28150 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:33.013569 28150 webserver.cc:460] Webserver started at http://127.0.0.1:40867/ using document root <none> and password file <none>
I20260809 00:46:33.013797 28150 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:33.013840 28150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:33.014129 28150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:33.015285 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "79e0c6b5b3764c4fb450470bbef31497"
format_stamp: "Formatted at 2026-08-09 00:46:33 on dist-test-slave-c12x"
server_key: "eeaffadb3b7c4ff6ea238c0a142a4680"
server_key_iv: "3aa41998a02e31791638a202b40e09e7"
server_key_version: "encryptionkey@0"
I20260809 00:46:33.015724 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "79e0c6b5b3764c4fb450470bbef31497"
format_stamp: "Formatted at 2026-08-09 00:46:33 on dist-test-slave-c12x"
server_key: "eeaffadb3b7c4ff6ea238c0a142a4680"
server_key_iv: "3aa41998a02e31791638a202b40e09e7"
server_key_version: "encryptionkey@0"
I20260809 00:46:33.017220 28150 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:33.018028 28170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:33.018252 28150 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:33.018347 28150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "79e0c6b5b3764c4fb450470bbef31497"
format_stamp: "Formatted at 2026-08-09 00:46:33 on dist-test-slave-c12x"
server_key: "eeaffadb3b7c4ff6ea238c0a142a4680"
server_key_iv: "3aa41998a02e31791638a202b40e09e7"
server_key_version: "encryptionkey@0"
I20260809 00:46:33.018409 28150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:33.035874 28150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:33.038342 28150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:33.038471 28150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:33.042831 28150 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39473
I20260809 00:46:33.042912 28244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39473 every 8 connection(s)
I20260809 00:46:33.043282 28150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:33.044329 28245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:33.046809 28245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497: Bootstrap starting.
I20260809 00:46:33.047479 28245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:33.047796 28245 log.cc:824] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:33.048673 28245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497: No bootstrap required, opened a new log
I20260809 00:46:33.050354 28245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e0c6b5b3764c4fb450470bbef31497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39473 } }
I20260809 00:46:33.050544 28245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:33.050596 28245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e0c6b5b3764c4fb450470bbef31497, State: Initialized, Role: FOLLOWER
I20260809 00:46:33.050629 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 28150
I20260809 00:46:33.050760 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:33.050798 28245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e0c6b5b3764c4fb450470bbef31497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39473 } }
I20260809 00:46:33.050897 28245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:33.050952 28245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:33.051004 28245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:33.051008 31139 external_mini_cluster.cc:1428] Setting key c485d0f1115665dcc009a6203e006caa
I20260809 00:46:33.052374 28245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e0c6b5b3764c4fb450470bbef31497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39473 } }
I20260809 00:46:33.052518 28245 leader_election.cc:302] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79e0c6b5b3764c4fb450470bbef31497; no voters:
I20260809 00:46:33.052701 28245 leader_election.cc:288] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:33.052737 28251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:33.052953 28251 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [term 1 LEADER]: Becoming Leader. State: Replica: 79e0c6b5b3764c4fb450470bbef31497, State: Running, Role: LEADER
I20260809 00:46:33.052985 28245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:33.053107 28251 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e0c6b5b3764c4fb450470bbef31497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39473 } }
I20260809 00:46:33.053647 28252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79e0c6b5b3764c4fb450470bbef31497. Latest consensus state: current_term: 1 leader_uuid: "79e0c6b5b3764c4fb450470bbef31497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e0c6b5b3764c4fb450470bbef31497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39473 } } }
I20260809 00:46:33.053745 28252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:33.053974 28254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79e0c6b5b3764c4fb450470bbef31497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79e0c6b5b3764c4fb450470bbef31497" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39473 } } }
I20260809 00:46:33.054337 28254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:33.054595 28263 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:33.055203 28263 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:33.057202 28263 catalog_manager.cc:1347] Generated new cluster ID: d51c22b4803a4b79be1efd2973e40ebd
I20260809 00:46:33.057298 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:33.051650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58271 (local address 127.0.0.1:39473)
0809 00:46:33.051866 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:33.051875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:33.051916 (+ 41us) server_negotiation.cc:407] Connection header received
0809 00:46:33.052023 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:33.052029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:33.052072 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:33.052198 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:33.052547 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:33.053799 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:33.054960 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:33.055603 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:33.055730 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:33.056556 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:33.056560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:33.056561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:33.056800 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:33.056832 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:33.056844 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:33.056946 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:33.057041 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 00:46:33.057073 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260809 00:46:33.057309 28263 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:33.058327 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:33.064044 28250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:33.060254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33235 (local address 127.0.0.1:39473)
0809 00:46:33.060291 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:33.060293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:33.060323 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:46:33.060402 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:33.060403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:33.060412 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:33.060470 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:33.060624 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:33.061372 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:33.062497 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:33.062804 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:33.062867 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:33.063749 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:33.063750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:33.063750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:33.063755 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:33.063769 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:33.063773 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:33.063828 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:33.063947 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 00:46:33.063960 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:46:33.069190 28263 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:33.069825 28263 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:33.076359 28263 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79e0c6b5b3764c4fb450470bbef31497: Generated new TSK 0
I20260809 00:46:33.076603 28263 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:33.080716 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 28150
W20260809 00:46:33.093449 28126 connection.cc:535] client connection to 127.0.0.1:39473 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39473): Connection reset by peer (error 104)
2026-08-09T00:46:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:33 dist-test-slave-c12x krb5kdc[28287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:33 dist-test-slave-c12x krb5kdc[28287](info): set up 2 sockets
Aug 09 00:46:33 dist-test-slave-c12x krb5kdc[28287](info): commencing operation
krb5kdc: starting...
W20260809 00:46:34.313290 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:34 dist-test-slave-c12x krb5kdc[28287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:34.438585 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:34.522755 28306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:34.522972 28306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:34.522995 28306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:34.524178 28306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:34.524223 28306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:34.524237 28306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:34.524250 28306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:34.524262 28306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:34.525717 28306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35937
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:34.525919 28306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:34.526117 28306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:34.531169 28312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:34.531169 28311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:34.531199 28314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:34.531478 28306 server_base.cc:1034] running on GCE node
I20260809 00:46:34.531736 28306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:34.532019 28306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:34.533179 28306 hybrid_clock.cc:648] HybridClock initialized: now 1786236394533163 us; error 29 us; skew 500 ppm
Aug 09 00:46:34 dist-test-slave-c12x krb5kdc[28287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236394, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:34.535986 28306 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:34.536387 28306 webserver.cc:460] Webserver started at http://127.0.0.1:39625/ using document root <none> and password file <none>
I20260809 00:46:34.536604 28306 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:34.536646 28306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:34.536738 28306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:34.537783 28306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "0faad314b1b54054b9918f7b44d1f3ff"
format_stamp: "Formatted at 2026-08-09 00:46:34 on dist-test-slave-c12x"
server_key: "4b8426490bc21297663d3901a9715209"
server_key_iv: "3b1f451d2b58139e8d7ac9cdbee00799"
server_key_version: "encryptionkey@0"
I20260809 00:46:34.538091 28306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "0faad314b1b54054b9918f7b44d1f3ff"
format_stamp: "Formatted at 2026-08-09 00:46:34 on dist-test-slave-c12x"
server_key: "4b8426490bc21297663d3901a9715209"
server_key_iv: "3b1f451d2b58139e8d7ac9cdbee00799"
server_key_version: "encryptionkey@0"
I20260809 00:46:34.539412 28306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:46:34.540247 28323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:34.540406 28306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:46:34.540525 28306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "0faad314b1b54054b9918f7b44d1f3ff"
format_stamp: "Formatted at 2026-08-09 00:46:34 on dist-test-slave-c12x"
server_key: "4b8426490bc21297663d3901a9715209"
server_key_iv: "3b1f451d2b58139e8d7ac9cdbee00799"
server_key_version: "encryptionkey@0"
I20260809 00:46:34.540608 28306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:34.543983 28306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:34.546334 28306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:34.546500 28306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:34.550482 28306 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35937
I20260809 00:46:34.550532 28400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35937 every 8 connection(s)
I20260809 00:46:34.550918 28306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:34.552078 28401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:34.554479 28401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff: Bootstrap starting.
I20260809 00:46:34.554541 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 28306
I20260809 00:46:34.554663 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:34.554857 31139 external_mini_cluster.cc:1428] Setting key 61ae0c6321e838bd4c17132b835b7823
I20260809 00:46:34.555264 28401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:34.555670 28401 log.cc:824] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:34.556469 28401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff: No bootstrap required, opened a new log
I20260809 00:46:34.557423 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:34.555401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48919 (local address 127.0.0.1:35937)
0809 00:46:34.555604 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:34.555611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:34.555637 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:46:34.556131 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:34.556139 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:34.556166 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:34.556290 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:34.556955 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:34.556958 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:34.556958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:34.556962 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:34.557004 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:34.557015 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:34.557085 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:34.557160 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 00:46:34.557192 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260809 00:46:34.558471 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:34.559088 28401 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0faad314b1b54054b9918f7b44d1f3ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35937 } }
I20260809 00:46:34.559276 28401 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:34.559319 28401 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0faad314b1b54054b9918f7b44d1f3ff, State: Initialized, Role: FOLLOWER
I20260809 00:46:34.559485 28401 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0faad314b1b54054b9918f7b44d1f3ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35937 } }
I20260809 00:46:34.559597 28401 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:34.559638 28401 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:34.559700 28401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:34.560582 28401 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0faad314b1b54054b9918f7b44d1f3ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35937 } }
I20260809 00:46:34.560696 28401 leader_election.cc:302] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0faad314b1b54054b9918f7b44d1f3ff; no voters:
I20260809 00:46:34.560889 28401 leader_election.cc:288] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:34.560925 28414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:34.561106 28414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [term 1 LEADER]: Becoming Leader. State: Replica: 0faad314b1b54054b9918f7b44d1f3ff, State: Running, Role: LEADER
I20260809 00:46:34.561177 28401 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:34.561276 28406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:34.560174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52657 (local address 127.0.0.1:35937)
0809 00:46:34.560213 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:34.560215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:34.560247 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:46:34.560323 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:34.560324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:34.560327 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:34.560368 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:34.560986 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:34.560987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:34.560987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:34.560992 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:34.561008 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:34.561012 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:34.561044 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:34.561171 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 00:46:34.561185 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:46:34.561267 28414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0faad314b1b54054b9918f7b44d1f3ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35937 } }
I20260809 00:46:34.562072 28417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0faad314b1b54054b9918f7b44d1f3ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0faad314b1b54054b9918f7b44d1f3ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35937 } } }
I20260809 00:46:34.562119 28416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0faad314b1b54054b9918f7b44d1f3ff. Latest consensus state: current_term: 1 leader_uuid: "0faad314b1b54054b9918f7b44d1f3ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0faad314b1b54054b9918f7b44d1f3ff" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35937 } } }
I20260809 00:46:34.562254 28417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:34.562270 28416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:34.562603 28425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:34.563138 28425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:34.564810 28425 catalog_manager.cc:1347] Generated new cluster ID: af6aeab6ac4b4501bf502b5cdc3f07e3
I20260809 00:46:34.564872 28425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:34.568810 28425 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:34.568987 28425 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:34.586079 28425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0faad314b1b54054b9918f7b44d1f3ff: Generated new TSK 0
I20260809 00:46:34.586313 28425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:34.594702 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 28306
2026-08-09T00:46:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1514 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:34 dist-test-slave-c12x krb5kdc[28449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:34 dist-test-slave-c12x krb5kdc[28449](info): set up 2 sockets
Aug 09 00:46:34 dist-test-slave-c12x krb5kdc[28449](info): commencing operation
krb5kdc: starting...
W20260809 00:46:35.811611 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:35 dist-test-slave-c12x krb5kdc[28449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:35.942070 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35133
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:36.027426 28466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:36.027732 28466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:36.027776 28466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:36.029004 28466 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:36.029083 28466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:36.029142 28466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:36.029174 28466 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:36.029199 28466 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:36.030824 28466 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:36.031133 28466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:36.031397 28466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:36.036509 28471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:36.036523 28474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:36.036656 28472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:36.037226 28466 server_base.cc:1034] running on GCE node
I20260809 00:46:36.037483 28466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:36.037765 28466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:36.038950 28466 hybrid_clock.cc:648] HybridClock initialized: now 1786236396038941 us; error 44 us; skew 500 ppm
Aug 09 00:46:36 dist-test-slave-c12x krb5kdc[28449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:36.041826 28466 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:36.042366 28466 webserver.cc:460] Webserver started at http://127.0.0.1:36231/ using document root <none> and password file <none>
I20260809 00:46:36.042637 28466 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:36.042686 28466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:36.042841 28466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:36.044234 28466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "a76b03d657ed4e85833da889eaa9f935"
format_stamp: "Formatted at 2026-08-09 00:46:36 on dist-test-slave-c12x"
server_key: "bf46a8963755603d58d180bd06a0d2bc"
server_key_iv: "551ecbbd2a4306f83727fd0c176f1244"
server_key_version: "encryptionkey@0"
I20260809 00:46:36.044771 28466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "a76b03d657ed4e85833da889eaa9f935"
format_stamp: "Formatted at 2026-08-09 00:46:36 on dist-test-slave-c12x"
server_key: "bf46a8963755603d58d180bd06a0d2bc"
server_key_iv: "551ecbbd2a4306f83727fd0c176f1244"
server_key_version: "encryptionkey@0"
I20260809 00:46:36.046339 28466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:46:36.047269 28482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:36.047554 28466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:36.047701 28466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "a76b03d657ed4e85833da889eaa9f935"
format_stamp: "Formatted at 2026-08-09 00:46:36 on dist-test-slave-c12x"
server_key: "bf46a8963755603d58d180bd06a0d2bc"
server_key_iv: "551ecbbd2a4306f83727fd0c176f1244"
server_key_version: "encryptionkey@0"
I20260809 00:46:36.047791 28466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:36.055209 28466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:36.057693 28466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:36.057870 28466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:36.062356 28466 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35133
I20260809 00:46:36.062388 28561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35133 every 8 connection(s)
I20260809 00:46:36.062773 28466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:36.063825 28562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:36.066293 28562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935: Bootstrap starting.
I20260809 00:46:36.066963 28562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:36.067272 28562 log.cc:824] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:36.068069 28562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935: No bootstrap required, opened a new log
I20260809 00:46:36.068332 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 28466
I20260809 00:46:36.068451 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:36.068660 31139 external_mini_cluster.cc:1428] Setting key 956c82bc1d7f4a1772fbaa972c8af896
I20260809 00:46:36.070355 28562 raft_consensus.cc:348] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76b03d657ed4e85833da889eaa9f935" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260809 00:46:36.070544 28562 raft_consensus.cc:374] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:36.070593 28562 raft_consensus.cc:729] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a76b03d657ed4e85833da889eaa9f935, State: Initialized, Role: FOLLOWER
I20260809 00:46:36.070727 28562 consensus_queue.cc:260] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76b03d657ed4e85833da889eaa9f935" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260809 00:46:36.070832 28562 raft_consensus.cc:388] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:36.070878 28562 raft_consensus.cc:482] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:36.070932 28562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:36.072017 28562 raft_consensus.cc:504] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76b03d657ed4e85833da889eaa9f935" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260809 00:46:36.072156 28562 leader_election.cc:302] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a76b03d657ed4e85833da889eaa9f935; no voters:
I20260809 00:46:36.072348 28562 leader_election.cc:288] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:36.072373 28567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:36.072490 28567 raft_consensus.cc:686] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [term 1 LEADER]: Becoming Leader. State: Replica: a76b03d657ed4e85833da889eaa9f935, State: Running, Role: LEADER
I20260809 00:46:36.072580 28567 consensus_queue.cc:237] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76b03d657ed4e85833da889eaa9f935" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } }
I20260809 00:46:36.072656 28562 sys_catalog.cc:564] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:36.072956 28569 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a76b03d657ed4e85833da889eaa9f935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76b03d657ed4e85833da889eaa9f935" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } } }
I20260809 00:46:36.073055 28569 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:36.073235 28568 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a76b03d657ed4e85833da889eaa9f935. Latest consensus state: current_term: 1 leader_uuid: "a76b03d657ed4e85833da889eaa9f935" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76b03d657ed4e85833da889eaa9f935" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35133 } } }
I20260809 00:46:36.073308 28568 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:36.073424 28574 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:36.073947 28574 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:36.076433 28574 catalog_manager.cc:1347] Generated new cluster ID: 0a4d5df19867440ca4ff97fea9f60b87
I20260809 00:46:36.076490 28574 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 00:46:36 dist-test-slave-c12x krb5kdc[28449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:46:36.081583 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:36.069236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53789 (local address 127.0.0.1:35133)
0809 00:46:36.069450 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:36.069459 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:36.069491 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:46:36.070353 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:36.070362 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:36.070454 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:36.070649 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:36.071479 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:36.072692 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:36.073816 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:36.074486 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:36.074664 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:36.077603 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:36.077607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:36.077608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:36.077621 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:36.079733 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:36.080181 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:36.080184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:36.080184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:36.080312 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:36.080542 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:36.080544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:36.080544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:36.080852 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:36.081010 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:36.081203 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 00:46:36.081348 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":113,"threads_started":1}
I20260809 00:46:36.082970 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:36.086525 28574 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:36.087242 28574 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:36.092118 28566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:36.084612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37565 (local address 127.0.0.1:35133)
0809 00:46:36.084692 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:36.084694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:36.084791 (+ 97us) server_negotiation.cc:407] Connection header received
0809 00:46:36.084885 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:36.084886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:36.084897 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:36.084928 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:36.085645 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:36.086257 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:36.087504 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:36.087817 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:36.087853 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:36.089348 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:36.089350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:36.089350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:36.089359 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:36.090892 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:36.091263 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:36.091264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:36.091264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:36.091314 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:36.091517 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:36.091518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:36.091518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:36.091649 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:36.091720 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:36.091910 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 00:46:36.092012 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 00:46:36.093080 28574 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a76b03d657ed4e85833da889eaa9f935: Generated new TSK 0
I20260809 00:46:36.093287 28574 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:36.100481 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 28466
2026-08-09T00:46:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:36 dist-test-slave-c12x krb5kdc[28604](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:36 dist-test-slave-c12x krb5kdc[28604](info): set up 2 sockets
Aug 09 00:46:36 dist-test-slave-c12x krb5kdc[28604](info): commencing operation
krb5kdc: starting...
W20260809 00:46:37.320639 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:37 dist-test-slave-c12x krb5kdc[28604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236397, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:37.450229 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:37.534612 28619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:37.534888 28619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:37.534929 28619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:37.536149 28619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:37.536228 28619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:37.536254 28619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:37.536283 28619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:37.536314 28619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:37.537843 28619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38695
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:37.538159 28619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:37.538409 28619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:37.543424 28628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:37.543475 28629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:37.543484 28632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:37.543777 28619 server_base.cc:1034] running on GCE node
I20260809 00:46:37.544088 28619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:37.544436 28619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:37.545608 28619 hybrid_clock.cc:648] HybridClock initialized: now 1786236397545609 us; error 50 us; skew 500 ppm
Aug 09 00:46:37 dist-test-slave-c12x krb5kdc[28604](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236397, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:37.551326 28619 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:37.551878 28619 webserver.cc:460] Webserver started at http://127.0.0.1:35903/ using document root <none> and password file <none>
I20260809 00:46:37.552101 28619 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:37.552145 28619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:37.552239 28619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:37.553362 28619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "15c5aaa0ace34ffd926ddb9311891323"
format_stamp: "Formatted at 2026-08-09 00:46:37 on dist-test-slave-c12x"
server_key: "366641feb83532580bcd9066eab095ee"
server_key_iv: "3371e8024290a7c2653797fea1738d0d"
server_key_version: "encryptionkey@0"
I20260809 00:46:37.553678 28619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "15c5aaa0ace34ffd926ddb9311891323"
format_stamp: "Formatted at 2026-08-09 00:46:37 on dist-test-slave-c12x"
server_key: "366641feb83532580bcd9066eab095ee"
server_key_iv: "3371e8024290a7c2653797fea1738d0d"
server_key_version: "encryptionkey@0"
I20260809 00:46:37.554983 28619 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:46:37.555739 28642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:37.555958 28619 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:37.556048 28619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "15c5aaa0ace34ffd926ddb9311891323"
format_stamp: "Formatted at 2026-08-09 00:46:37 on dist-test-slave-c12x"
server_key: "366641feb83532580bcd9066eab095ee"
server_key_iv: "3371e8024290a7c2653797fea1738d0d"
server_key_version: "encryptionkey@0"
I20260809 00:46:37.556108 28619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:37.579797 28619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:37.582193 28619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:37.582311 28619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:37.586441 28619 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38695
I20260809 00:46:37.586505 28712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38695 every 8 connection(s)
I20260809 00:46:37.586881 28619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:37.587877 28713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:37.590269 28713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323: Bootstrap starting.
I20260809 00:46:37.590893 28713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:37.591212 28713 log.cc:824] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:37.592101 28713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323: No bootstrap required, opened a new log
I20260809 00:46:37.593739 28713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15c5aaa0ace34ffd926ddb9311891323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260809 00:46:37.593886 28713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:37.593909 28713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c5aaa0ace34ffd926ddb9311891323, State: Initialized, Role: FOLLOWER
I20260809 00:46:37.594025 28713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15c5aaa0ace34ffd926ddb9311891323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260809 00:46:37.594112 28713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:37.594151 28713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:37.594230 28713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:37.594794 28713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15c5aaa0ace34ffd926ddb9311891323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260809 00:46:37.594925 28713 leader_election.cc:302] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15c5aaa0ace34ffd926ddb9311891323; no voters:
I20260809 00:46:37.595131 28713 leader_election.cc:288] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:37.595299 28718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:37.595427 28713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:37.595536 28718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [term 1 LEADER]: Becoming Leader. State: Replica: 15c5aaa0ace34ffd926ddb9311891323, State: Running, Role: LEADER
I20260809 00:46:37.595643 28718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15c5aaa0ace34ffd926ddb9311891323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } }
I20260809 00:46:37.596252 28720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15c5aaa0ace34ffd926ddb9311891323. Latest consensus state: current_term: 1 leader_uuid: "15c5aaa0ace34ffd926ddb9311891323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15c5aaa0ace34ffd926ddb9311891323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260809 00:46:37.596354 28720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:37.596451 28719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15c5aaa0ace34ffd926ddb9311891323" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15c5aaa0ace34ffd926ddb9311891323" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38695 } } }
I20260809 00:46:37.596611 28719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:37.596716 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 28619
I20260809 00:46:37.596838 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:37.597155 31139 external_mini_cluster.cc:1428] Setting key 1c4c6bd4921f187221e7ba4cc09abfc4
W20260809 00:46:37.597666 28741 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:46:37.597891 28741 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:46:37.598064 28732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:37.598282 28732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:37.599860 28732 catalog_manager.cc:1347] Generated new cluster ID: 03444e477cdc4dc1b227332044624db7
I20260809 00:46:37.599936 28732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:37.603646 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:37.597695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37783 (local address 127.0.0.1:38695)
0809 00:46:37.598005 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:37.598014 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:37.598045 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:46:37.598168 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:37.598175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:37.598232 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:37.598629 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:37.598725 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:37.600147 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:37.601261 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:37.601894 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:37.602013 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:37.602759 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:37.602763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:37.602764 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:37.602980 (+ 216us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:37.603023 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:37.603041 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:37.603119 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:37.603198 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:46:37.603369 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260809 00:46:37.604610 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:37.609884 28732 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:37.610004 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:37.606186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41147 (local address 127.0.0.1:38695)
0809 00:46:37.606223 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:37.606225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:37.606342 (+ 117us) server_negotiation.cc:407] Connection header received
0809 00:46:37.606436 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:37.606437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:37.606447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:37.606476 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:37.606700 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:37.607458 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:37.608481 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:37.608779 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:37.608853 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:37.609658 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:37.609660 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:37.609661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:37.609666 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:37.609681 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:37.609687 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:37.609738 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:37.609824 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 00:46:37.609850 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:46:37.610780 28732 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:37.624902 28732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15c5aaa0ace34ffd926ddb9311891323: Generated new TSK 0
I20260809 00:46:37.625195 28732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:37.627573 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 28619
2026-08-09T00:46:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:37 dist-test-slave-c12x krb5kdc[28758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:37 dist-test-slave-c12x krb5kdc[28758](info): set up 2 sockets
Aug 09 00:46:37 dist-test-slave-c12x krb5kdc[28758](info): commencing operation
krb5kdc: starting...
W20260809 00:46:38.851347 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:38 dist-test-slave-c12x krb5kdc[28758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:38.981052 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36403
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:39.071782 28774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:39.072080 28774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:39.072124 28774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:39.073734 28774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:39.073827 28774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:39.073865 28774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:39.073895 28774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:39.073922 28774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:39.075650 28774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:39.075960 28774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:39.076212 28774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:39.081462 28783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:39.081492 28774 server_base.cc:1034] running on GCE node
W20260809 00:46:39.081483 28782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:39.081497 28788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:39.081961 28774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:39.082273 28774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:39.083503 28774 hybrid_clock.cc:648] HybridClock initialized: now 1786236399083488 us; error 49 us; skew 500 ppm
Aug 09 00:46:39 dist-test-slave-c12x krb5kdc[28758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236399, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:39.086171 28774 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:39.086660 28774 webserver.cc:460] Webserver started at http://127.0.0.1:45109/ using document root <none> and password file <none>
I20260809 00:46:39.086890 28774 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:39.086961 28774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:39.087146 28774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:39.088335 28774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "8ca4255dc5314f438865e71892ef87e3"
format_stamp: "Formatted at 2026-08-09 00:46:39 on dist-test-slave-c12x"
server_key: "dc1e218aa95844b51675826befedd12e"
server_key_iv: "2c44f9c81a4e208560d4ab283cb644dd"
server_key_version: "encryptionkey@0"
I20260809 00:46:39.088728 28774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "8ca4255dc5314f438865e71892ef87e3"
format_stamp: "Formatted at 2026-08-09 00:46:39 on dist-test-slave-c12x"
server_key: "dc1e218aa95844b51675826befedd12e"
server_key_iv: "2c44f9c81a4e208560d4ab283cb644dd"
server_key_version: "encryptionkey@0"
I20260809 00:46:39.090119 28774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:46:39.090925 28797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:39.091241 28774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:39.091355 28774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "8ca4255dc5314f438865e71892ef87e3"
format_stamp: "Formatted at 2026-08-09 00:46:39 on dist-test-slave-c12x"
server_key: "dc1e218aa95844b51675826befedd12e"
server_key_iv: "2c44f9c81a4e208560d4ab283cb644dd"
server_key_version: "encryptionkey@0"
I20260809 00:46:39.091434 28774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:39.094818 28774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:39.096866 28774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:39.097028 28774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:39.101325 28774 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36403
I20260809 00:46:39.101356 28875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36403 every 8 connection(s)
I20260809 00:46:39.101708 28774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:39.102728 28876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:39.105150 28876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3: Bootstrap starting.
I20260809 00:46:39.105777 28876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:39.106071 28876 log.cc:824] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:39.106808 28876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3: No bootstrap required, opened a new log
I20260809 00:46:39.107600 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 28774
I20260809 00:46:39.107692 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:39.107894 31139 external_mini_cluster.cc:1428] Setting key f6340ba083726e9f3c5fa841c5c7fb04
I20260809 00:46:39.108961 28876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4255dc5314f438865e71892ef87e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36403 } }
I20260809 00:46:39.109164 28876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:39.109253 28876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ca4255dc5314f438865e71892ef87e3, State: Initialized, Role: FOLLOWER
I20260809 00:46:39.109431 28876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4255dc5314f438865e71892ef87e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36403 } }
I20260809 00:46:39.109562 28876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:39.109614 28876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:39.109671 28876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:39.110368 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:39.108411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37119 (local address 127.0.0.1:36403)
0809 00:46:39.108637 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:39.108644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:39.108670 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:46:39.108936 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:39.108942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:39.108968 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:39.109085 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:39.109935 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:39.109939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:39.109940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:39.109944 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:39.109993 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:39.110004 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:39.110083 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:39.110155 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 00:46:39.110198 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":100,"threads_started":1}
I20260809 00:46:39.110791 28876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4255dc5314f438865e71892ef87e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36403 } }
I20260809 00:46:39.110924 28876 leader_election.cc:302] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ca4255dc5314f438865e71892ef87e3; no voters:
I20260809 00:46:39.111101 28876 leader_election.cc:288] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:39.111245 28884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:39.111331 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:39.111392 28876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:39.111536 28884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [term 1 LEADER]: Becoming Leader. State: Replica: 8ca4255dc5314f438865e71892ef87e3, State: Running, Role: LEADER
I20260809 00:46:39.111624 28884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4255dc5314f438865e71892ef87e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36403 } }
I20260809 00:46:39.111970 28886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ca4255dc5314f438865e71892ef87e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4255dc5314f438865e71892ef87e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36403 } } }
I20260809 00:46:39.112052 28886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:39.111984 28887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ca4255dc5314f438865e71892ef87e3. Latest consensus state: current_term: 1 leader_uuid: "8ca4255dc5314f438865e71892ef87e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ca4255dc5314f438865e71892ef87e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36403 } } }
I20260809 00:46:39.112093 28887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:39.112366 28898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:39.112921 28898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:39.114583 28898 catalog_manager.cc:1347] Generated new cluster ID: 2a0445d130d04051964eaa442bbd8321
I20260809 00:46:39.114629 28898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:39.117038 28882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:39.115859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49185 (local address 127.0.0.1:36403)
0809 00:46:39.115900 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:39.115902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:39.116059 (+ 157us) server_negotiation.cc:407] Connection header received
0809 00:46:39.116124 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:39.116125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:39.116131 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:39.116188 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:39.116816 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:39.116818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:39.116818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:39.116822 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:39.116837 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:39.116841 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:39.116893 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:39.116963 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 00:46:39.116976 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 00:46:39.121935 28898 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:39.122128 28898 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:39.129392 28898 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ca4255dc5314f438865e71892ef87e3: Generated new TSK 0
I20260809 00:46:39.129580 28898 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:39.131567 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 28774
2026-08-09T00:46:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1500 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:39 dist-test-slave-c12x krb5kdc[28920](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:39 dist-test-slave-c12x krb5kdc[28920](info): set up 2 sockets
Aug 09 00:46:39 dist-test-slave-c12x krb5kdc[28920](info): commencing operation
krb5kdc: starting...
W20260809 00:46:40.368394 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:40 dist-test-slave-c12x krb5kdc[28920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:40.500083 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:40.584692 28936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:40.584935 28936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:40.584962 28936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:40.586258 28936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:40.586313 28936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:40.586328 28936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:40.586341 28936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:40.586354 28936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:40.588037 28936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:40.588353 28936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:40.588644 28936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:40.594208 28947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:40.594219 28945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:40.594234 28949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:40.594316 28936 server_base.cc:1034] running on GCE node
I20260809 00:46:40.594774 28936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:40.595175 28936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:40.596360 28936 hybrid_clock.cc:648] HybridClock initialized: now 1786236400596289 us; error 125 us; skew 500 ppm
Aug 09 00:46:40 dist-test-slave-c12x krb5kdc[28920](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236400, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:40.599130 28936 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:40.599682 28936 webserver.cc:460] Webserver started at http://127.0.0.1:42409/ using document root <none> and password file <none>
I20260809 00:46:40.599885 28936 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:40.599928 28936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:40.600034 28936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:40.601154 28936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "eba29c785ba84842964ad0e33d5bc3e6"
format_stamp: "Formatted at 2026-08-09 00:46:40 on dist-test-slave-c12x"
server_key: "eb41045c51a29661697627c31abfe63f"
server_key_iv: "4ee24da713d7d20fe7472cb377e69d4e"
server_key_version: "encryptionkey@0"
I20260809 00:46:40.601480 28936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "eba29c785ba84842964ad0e33d5bc3e6"
format_stamp: "Formatted at 2026-08-09 00:46:40 on dist-test-slave-c12x"
server_key: "eb41045c51a29661697627c31abfe63f"
server_key_iv: "4ee24da713d7d20fe7472cb377e69d4e"
server_key_version: "encryptionkey@0"
I20260809 00:46:40.602955 28936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:40.603847 28959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:40.604087 28936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:40.604187 28936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "eba29c785ba84842964ad0e33d5bc3e6"
format_stamp: "Formatted at 2026-08-09 00:46:40 on dist-test-slave-c12x"
server_key: "eb41045c51a29661697627c31abfe63f"
server_key_iv: "4ee24da713d7d20fe7472cb377e69d4e"
server_key_version: "encryptionkey@0"
I20260809 00:46:40.604252 28936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:40.627588 28936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:40.630098 28936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:40.630225 28936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:40.634279 28936 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35507
I20260809 00:46:40.634699 28936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:40.634689 29037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35507 every 8 connection(s)
I20260809 00:46:40.635548 29039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:40.636206 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 28936
I20260809 00:46:40.636339 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:40.636605 31139 external_mini_cluster.cc:1428] Setting key c16b2e767b88bc4b435c0de93095cc15
I20260809 00:46:40.638537 29039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6: Bootstrap starting.
I20260809 00:46:40.639349 29039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:40.639698 29039 log.cc:824] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:40.640522 29039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6: No bootstrap required, opened a new log
I20260809 00:46:40.642930 29039 raft_consensus.cc:348] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba29c785ba84842964ad0e33d5bc3e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } }
I20260809 00:46:40.643095 29039 raft_consensus.cc:374] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:40.643119 29039 raft_consensus.cc:729] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eba29c785ba84842964ad0e33d5bc3e6, State: Initialized, Role: FOLLOWER
I20260809 00:46:40.643280 29039 consensus_queue.cc:260] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba29c785ba84842964ad0e33d5bc3e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } }
I20260809 00:46:40.643375 29039 raft_consensus.cc:388] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:40.643417 29039 raft_consensus.cc:482] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:40.643532 29039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 00:46:40 dist-test-slave-c12x krb5kdc[28920](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:46:40.644428 29039 raft_consensus.cc:504] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba29c785ba84842964ad0e33d5bc3e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } }
I20260809 00:46:40.644601 29039 leader_election.cc:302] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eba29c785ba84842964ad0e33d5bc3e6; no voters:
I20260809 00:46:40.644868 29039 leader_election.cc:288] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:40.644903 29046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:40.645062 29046 raft_consensus.cc:686] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [term 1 LEADER]: Becoming Leader. State: Replica: eba29c785ba84842964ad0e33d5bc3e6, State: Running, Role: LEADER
I20260809 00:46:40.645190 29046 consensus_queue.cc:237] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba29c785ba84842964ad0e33d5bc3e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } }
I20260809 00:46:40.645229 29039 sys_catalog.cc:564] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:40.645677 29047 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eba29c785ba84842964ad0e33d5bc3e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba29c785ba84842964ad0e33d5bc3e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } } }
I20260809 00:46:40.646868 29047 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:40.645810 29048 sys_catalog.cc:455] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eba29c785ba84842964ad0e33d5bc3e6. Latest consensus state: current_term: 1 leader_uuid: "eba29c785ba84842964ad0e33d5bc3e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eba29c785ba84842964ad0e33d5bc3e6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } } }
I20260809 00:46:40.647269 29048 sys_catalog.cc:458] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6 [sys.catalog]: This master's current role is: LEADER
W20260809 00:46:40.647360 29067 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:46:40.647425 29067 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:46:40.647550 29069 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:40.647727 29069 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:40.648810 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:40.637221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:46365 (local address 127.0.0.1:35507)
0809 00:46:40.637409 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:40.637418 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:40.637443 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:46:40.637862 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:40.637870 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:40.637920 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:40.638076 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:40.638754 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:40.640203 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:40.641036 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:40.641703 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:40.641831 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:40.644665 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:40.644669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:40.644670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:40.644700 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:40.646717 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:40.647227 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:40.647231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:40.647232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:40.647369 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:40.647723 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:40.647725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:40.647725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:40.648094 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:40.648257 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:40.648468 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 00:46:40.648591 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260809 00:46:40.649957 29069 catalog_manager.cc:1347] Generated new cluster ID: 45e5be957afd40b185d28285586f12e4
I20260809 00:46:40.649983 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:40.650013 29069 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:40.664057 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:40.651851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:41127 (local address 127.0.0.1:35507)
0809 00:46:40.651886 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:40.651888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:40.655726 (+ 3838us) server_negotiation.cc:407] Connection header received
0809 00:46:40.655805 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:40.655806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:40.655822 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:40.655902 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:40.656679 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:40.657487 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:40.658627 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:40.659162 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:40.659243 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:40.660942 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:40.660943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:40.660944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:40.660953 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:40.662639 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:40.663125 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:40.663126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:40.663126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:40.663190 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:40.663455 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:40.663456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:40.663456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:40.663602 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:40.663703 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:40.663872 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 00:46:40.663952 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 00:46:40.667768 29069 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:40.668548 29069 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:40.671859 29069 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eba29c785ba84842964ad0e33d5bc3e6: Generated new TSK 0
I20260809 00:46:40.672089 29069 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:40.674659 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 28936
W20260809 00:46:40.687898 28916 connection.cc:535] client connection to 127.0.0.1:35507 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35507): Connection reset by peer (error 104)
2026-08-09T00:46:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:40 dist-test-slave-c12x krb5kdc[29082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:40 dist-test-slave-c12x krb5kdc[29082](info): set up 2 sockets
Aug 09 00:46:40 dist-test-slave-c12x krb5kdc[29082](info): commencing operation
krb5kdc: starting...
W20260809 00:46:41.920790 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:42 dist-test-slave-c12x krb5kdc[29082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:42.054462 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:42.139412 29107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:42.139715 29107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:42.139756 29107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:42.140959 29107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:42.141049 29107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:42.141077 29107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:42.141110 29107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:42.141136 29107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:42.142656 29107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:42.142966 29107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:42.143213 29107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:42.148376 29114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:42.148341 29118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:42.148458 29107 server_base.cc:1034] running on GCE node
W20260809 00:46:42.148360 29115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:42.148902 29107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:42.149236 29107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:42.150406 29107 hybrid_clock.cc:648] HybridClock initialized: now 1786236402150391 us; error 45 us; skew 500 ppm
Aug 09 00:46:42 dist-test-slave-c12x krb5kdc[29082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236402, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:42.153056 29107 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:42.153532 29107 webserver.cc:460] Webserver started at http://127.0.0.1:43721/ using document root <none> and password file <none>
I20260809 00:46:42.153784 29107 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:42.153854 29107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:42.154008 29107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:42.155159 29107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "3cf709ebbc71432da3ebe0b6c1994a53"
format_stamp: "Formatted at 2026-08-09 00:46:42 on dist-test-slave-c12x"
server_key: "e7717af9b95ef045c36bfeaffd6e8d71"
server_key_iv: "289aa2b7c56c6c5473d7532ead7da9a9"
server_key_version: "encryptionkey@0"
I20260809 00:46:42.155617 29107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "3cf709ebbc71432da3ebe0b6c1994a53"
format_stamp: "Formatted at 2026-08-09 00:46:42 on dist-test-slave-c12x"
server_key: "e7717af9b95ef045c36bfeaffd6e8d71"
server_key_iv: "289aa2b7c56c6c5473d7532ead7da9a9"
server_key_version: "encryptionkey@0"
I20260809 00:46:42.157097 29107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:42.157976 29126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:42.158165 29107 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:42.158286 29107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "3cf709ebbc71432da3ebe0b6c1994a53"
format_stamp: "Formatted at 2026-08-09 00:46:42 on dist-test-slave-c12x"
server_key: "e7717af9b95ef045c36bfeaffd6e8d71"
server_key_iv: "289aa2b7c56c6c5473d7532ead7da9a9"
server_key_version: "encryptionkey@0"
I20260809 00:46:42.158370 29107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:42.187850 29107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:42.190135 29107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:42.190335 29107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:42.194728 29107 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39799
I20260809 00:46:42.194772 29204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39799 every 8 connection(s)
I20260809 00:46:42.195168 29107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:42.196214 29205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:42.198535 29205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53: Bootstrap starting.
I20260809 00:46:42.199095 29205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:42.199330 29205 log.cc:824] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:42.200120 29205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53: No bootstrap required, opened a new log
I20260809 00:46:42.201076 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 29107
I20260809 00:46:42.201223 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:42.201411 31139 external_mini_cluster.cc:1428] Setting key cd5b50d39374da6fe941d485d744a75b
I20260809 00:46:42.202119 29205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf709ebbc71432da3ebe0b6c1994a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } }
I20260809 00:46:42.202345 29205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:42.202386 29205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf709ebbc71432da3ebe0b6c1994a53, State: Initialized, Role: FOLLOWER
I20260809 00:46:42.202517 29205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf709ebbc71432da3ebe0b6c1994a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } }
I20260809 00:46:42.202613 29205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:42.202701 29205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:42.202759 29205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:42.203408 29205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf709ebbc71432da3ebe0b6c1994a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } }
I20260809 00:46:42.203568 29205 leader_election.cc:302] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cf709ebbc71432da3ebe0b6c1994a53; no voters:
I20260809 00:46:42.203789 29205 leader_election.cc:288] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:42.203815 29214 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:42.203933 29214 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [term 1 LEADER]: Becoming Leader. State: Replica: 3cf709ebbc71432da3ebe0b6c1994a53, State: Running, Role: LEADER
I20260809 00:46:42.204110 29205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:42.204078 29214 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf709ebbc71432da3ebe0b6c1994a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } }
I20260809 00:46:42.204586 29216 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cf709ebbc71432da3ebe0b6c1994a53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf709ebbc71432da3ebe0b6c1994a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } } }
I20260809 00:46:42.204746 29216 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:42.204612 29217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cf709ebbc71432da3ebe0b6c1994a53. Latest consensus state: current_term: 1 leader_uuid: "3cf709ebbc71432da3ebe0b6c1994a53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf709ebbc71432da3ebe0b6c1994a53" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39799 } } }
I20260809 00:46:42.204962 29230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:42.204984 29217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:42.205564 29230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:42.207640 29230 catalog_manager.cc:1347] Generated new cluster ID: 8ec23b324cef4f1d975ae6e0d4eb1df4
I20260809 00:46:42.207697 29230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:42.207907 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:42.202015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:34769 (local address 127.0.0.1:39799)
0809 00:46:42.202242 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:42.202252 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:42.202287 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:46:42.202420 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:42.202428 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:42.202502 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:42.202659 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:42.202892 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:42.204409 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:42.205506 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:42.206120 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:42.206257 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:42.207033 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:42.207037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:42.207037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:42.207284 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:42.207315 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:42.207327 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:42.207434 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:42.207656 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 00:46:42.207688 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":116,"threads_started":1}
I20260809 00:46:42.208870 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:42.213636 29213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:42.210311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:32925 (local address 127.0.0.1:39799)
0809 00:46:42.210356 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:42.210357 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:42.210477 (+ 120us) server_negotiation.cc:407] Connection header received
0809 00:46:42.210552 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:42.210553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:42.210565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:42.210622 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:42.210774 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:42.211245 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:42.212275 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:42.212526 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:42.212610 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:42.213357 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:42.213359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:42.213359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:42.213364 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:42.213379 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:42.213384 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:42.213440 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:42.213517 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 00:46:42.213531 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:46:42.218128 29230 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:42.218946 29230 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:42.222150 29230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cf709ebbc71432da3ebe0b6c1994a53: Generated new TSK 0
I20260809 00:46:42.222347 29230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:42.230860 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 29107
W20260809 00:46:42.242479 29078 connection.cc:535] client connection to 127.0.0.1:39799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39799): Connection reset by peer (error 104)
2026-08-09T00:46:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:42 dist-test-slave-c12x krb5kdc[29254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:42 dist-test-slave-c12x krb5kdc[29254](info): set up 2 sockets
Aug 09 00:46:42 dist-test-slave-c12x krb5kdc[29254](info): commencing operation
krb5kdc: starting...
W20260809 00:46:43.487847 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:43 dist-test-slave-c12x krb5kdc[29254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:43.624840 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:43.711422 29269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:43.711743 29269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:43.711788 29269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:43.713047 29269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:43.713140 29269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:43.713164 29269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:43.713195 29269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:43.713228 29269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:43.714848 29269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:43.715185 29269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:43.715468 29269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:43.720906 29278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:43.720901 29281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:43.720901 29277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:43.721060 29269 server_base.cc:1034] running on GCE node
I20260809 00:46:43.721379 29269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:43.721724 29269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:43.722884 29269 hybrid_clock.cc:648] HybridClock initialized: now 1786236403722855 us; error 52 us; skew 500 ppm
Aug 09 00:46:43 dist-test-slave-c12x krb5kdc[29254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236403, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:43.725651 29269 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:43.726166 29269 webserver.cc:460] Webserver started at http://127.0.0.1:43125/ using document root <none> and password file <none>
I20260809 00:46:43.726414 29269 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:43.726486 29269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:43.726643 29269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:43.727849 29269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "8677e58c0d044105813d7d57f0844798"
format_stamp: "Formatted at 2026-08-09 00:46:43 on dist-test-slave-c12x"
server_key: "ec78436a6d9ca91c64d0447ff953ca0c"
server_key_iv: "2166aef0ec687f4ffab6299de942970c"
server_key_version: "encryptionkey@0"
I20260809 00:46:43.728240 29269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "8677e58c0d044105813d7d57f0844798"
format_stamp: "Formatted at 2026-08-09 00:46:43 on dist-test-slave-c12x"
server_key: "ec78436a6d9ca91c64d0447ff953ca0c"
server_key_iv: "2166aef0ec687f4ffab6299de942970c"
server_key_version: "encryptionkey@0"
I20260809 00:46:43.729697 29269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:46:43.730558 29292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:43.730780 29269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:43.730896 29269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "8677e58c0d044105813d7d57f0844798"
format_stamp: "Formatted at 2026-08-09 00:46:43 on dist-test-slave-c12x"
server_key: "ec78436a6d9ca91c64d0447ff953ca0c"
server_key_iv: "2166aef0ec687f4ffab6299de942970c"
server_key_version: "encryptionkey@0"
I20260809 00:46:43.730978 29269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:43.734756 29269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:43.737071 29269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:43.737250 29269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:43.742095 29269 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35799
I20260809 00:46:43.742126 29369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35799 every 8 connection(s)
I20260809 00:46:43.742493 29269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:43.743498 29371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:43.745872 29371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798: Bootstrap starting.
I20260809 00:46:43.746495 29371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:43.746778 29371 log.cc:824] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:43.747653 29371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798: No bootstrap required, opened a new log
I20260809 00:46:43.749253 29371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8677e58c0d044105813d7d57f0844798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } }
I20260809 00:46:43.749401 29371 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:43.749426 29371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8677e58c0d044105813d7d57f0844798, State: Initialized, Role: FOLLOWER
I20260809 00:46:43.749576 29371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8677e58c0d044105813d7d57f0844798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } }
I20260809 00:46:43.749657 29371 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:43.749702 29371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:43.749774 29371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:43.750337 29371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8677e58c0d044105813d7d57f0844798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } }
I20260809 00:46:43.750460 29371 leader_election.cc:302] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8677e58c0d044105813d7d57f0844798; no voters:
I20260809 00:46:43.750654 29371 leader_election.cc:288] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:43.750708 29374 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:43.750895 29374 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [term 1 LEADER]: Becoming Leader. State: Replica: 8677e58c0d044105813d7d57f0844798, State: Running, Role: LEADER
I20260809 00:46:43.750942 29371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:43.751061 29374 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8677e58c0d044105813d7d57f0844798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } }
I20260809 00:46:43.751394 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 29269
I20260809 00:46:43.751508 29376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8677e58c0d044105813d7d57f0844798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8677e58c0d044105813d7d57f0844798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } } }
I20260809 00:46:43.751479 29377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8677e58c0d044105813d7d57f0844798. Latest consensus state: current_term: 1 leader_uuid: "8677e58c0d044105813d7d57f0844798" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8677e58c0d044105813d7d57f0844798" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35799 } } }
I20260809 00:46:43.751574 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:43.751567 29376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:43.751574 29377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:43.751781 31139 external_mini_cluster.cc:1428] Setting key c652694047b683364efa6e55d379e026
I20260809 00:46:43.751778 29381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:43.752288 29381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:43.754128 29381 catalog_manager.cc:1347] Generated new cluster ID: 97f9cd279bf045c58da784fadd8a224f
I20260809 00:46:43.754206 29381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:43.754460 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:43.752605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:37745 (local address 127.0.0.1:35799)
0809 00:46:43.752764 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:43.752819 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:43.752851 (+ 32us) server_negotiation.cc:407] Connection header received
0809 00:46:43.753019 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:43.753029 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:43.753072 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:43.753231 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:43.753912 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:43.753916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:43.753916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:43.753920 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:43.753967 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:43.753982 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:43.754206 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:43.754239 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 00:46:43.754284 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260809 00:46:43.755347 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:43.757741 29401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:43.756857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:58929 (local address 127.0.0.1:35799)
0809 00:46:43.756892 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:43.756894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:43.756991 (+ 97us) server_negotiation.cc:407] Connection header received
0809 00:46:43.757064 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:43.757065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:43.757069 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:43.757119 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:43.757580 (+ 461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:43.757581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:43.757581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:43.757584 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:43.757596 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:43.757600 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:43.757642 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:43.757688 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 00:46:43.757700 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:46:43.784258 29381 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:43.784457 29381 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:43.787825 29381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8677e58c0d044105813d7d57f0844798: Generated new TSK 0
I20260809 00:46:43.788690 29381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:43.796401 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 29269
2026-08-09T00:46:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1565 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:43 dist-test-slave-c12x krb5kdc[29414](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:43 dist-test-slave-c12x krb5kdc[29414](info): set up 2 sockets
Aug 09 00:46:43 dist-test-slave-c12x krb5kdc[29414](info): commencing operation
krb5kdc: starting...
W20260809 00:46:45.041318 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:45 dist-test-slave-c12x krb5kdc[29414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:45.174244 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:45.261018 29430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:45.261260 29430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:45.261286 29430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:45.263198 29430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:45.263270 29430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:45.263298 29430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:45.263322 29430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:45.263346 29430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:45.265585 29430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36187
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:45.265934 29430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:45.266193 29430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:45.271723 29439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:45.271735 29442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:45.271852 29430 server_base.cc:1034] running on GCE node
W20260809 00:46:45.271705 29438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:45.272224 29430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:45.272549 29430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:45.273722 29430 hybrid_clock.cc:648] HybridClock initialized: now 1786236405273708 us; error 30 us; skew 500 ppm
Aug 09 00:46:45 dist-test-slave-c12x krb5kdc[29414](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236405, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:45.276605 29430 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:45.277148 29430 webserver.cc:460] Webserver started at http://127.0.0.1:45603/ using document root <none> and password file <none>
I20260809 00:46:45.277395 29430 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:45.277467 29430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:45.277621 29430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:45.278764 29430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "8f84ee81dff446e29e63677ac592c010"
format_stamp: "Formatted at 2026-08-09 00:46:45 on dist-test-slave-c12x"
server_key: "165510d09502a49530b463a9cbac60f5"
server_key_iv: "c2389d2104cbe0815f1cab340d2d0ab9"
server_key_version: "encryptionkey@0"
I20260809 00:46:45.279282 29430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "8f84ee81dff446e29e63677ac592c010"
format_stamp: "Formatted at 2026-08-09 00:46:45 on dist-test-slave-c12x"
server_key: "165510d09502a49530b463a9cbac60f5"
server_key_iv: "c2389d2104cbe0815f1cab340d2d0ab9"
server_key_version: "encryptionkey@0"
I20260809 00:46:45.280807 29430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:45.281618 29452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:45.281802 29430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:45.281926 29430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "8f84ee81dff446e29e63677ac592c010"
format_stamp: "Formatted at 2026-08-09 00:46:45 on dist-test-slave-c12x"
server_key: "165510d09502a49530b463a9cbac60f5"
server_key_iv: "c2389d2104cbe0815f1cab340d2d0ab9"
server_key_version: "encryptionkey@0"
I20260809 00:46:45.282011 29430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:45.303030 29430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:45.305629 29430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:45.305815 29430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:45.310854 29430 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36187
I20260809 00:46:45.310920 29531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36187 every 8 connection(s)
I20260809 00:46:45.311347 29430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:45.312755 29534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:45.316637 29534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010: Bootstrap starting.
I20260809 00:46:45.317416 29534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:45.317759 29534 log.cc:824] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:45.318662 29534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010: No bootstrap required, opened a new log
I20260809 00:46:45.320551 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 29430
I20260809 00:46:45.320562 29534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f84ee81dff446e29e63677ac592c010" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } }
I20260809 00:46:45.320680 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:45.320708 29534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:45.320730 29534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f84ee81dff446e29e63677ac592c010, State: Initialized, Role: FOLLOWER
I20260809 00:46:45.320914 31139 external_mini_cluster.cc:1428] Setting key 3c7f3afabf288ebf1a9e4983e1864adf
I20260809 00:46:45.320879 29534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f84ee81dff446e29e63677ac592c010" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } }
I20260809 00:46:45.320953 29534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:45.320977 29534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:45.321008 29534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:45.321559 29534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f84ee81dff446e29e63677ac592c010" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } }
I20260809 00:46:45.321694 29534 leader_election.cc:302] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f84ee81dff446e29e63677ac592c010; no voters:
I20260809 00:46:45.321898 29534 leader_election.cc:288] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:45.321930 29541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:45.322153 29541 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [term 1 LEADER]: Becoming Leader. State: Replica: 8f84ee81dff446e29e63677ac592c010, State: Running, Role: LEADER
I20260809 00:46:45.322196 29534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:45.322341 29541 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f84ee81dff446e29e63677ac592c010" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } }
I20260809 00:46:45.322779 29541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f84ee81dff446e29e63677ac592c010. Latest consensus state: current_term: 1 leader_uuid: "8f84ee81dff446e29e63677ac592c010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f84ee81dff446e29e63677ac592c010" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } } }
I20260809 00:46:45.323045 29541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:45.322947 29545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f84ee81dff446e29e63677ac592c010" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f84ee81dff446e29e63677ac592c010" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36187 } } }
I20260809 00:46:45.323316 29545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:45.323753 29553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:45.324337 29553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:45.329833 29553 catalog_manager.cc:1347] Generated new cluster ID: 6ee57084b6284ee89d40e65073ef1122
I20260809 00:46:45.329924 29553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:45.337932 29553 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:45.338872 29553 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 00:46:45 dist-test-slave-c12x krb5kdc[29414](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 00:46:45.346577 29553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f84ee81dff446e29e63677ac592c010: Generated new TSK 0
I20260809 00:46:45.347092 29553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:45.350356 29542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:45.321824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:52957 (local address 127.0.0.1:36187)
0809 00:46:45.322085 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:45.322098 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:45.327813 (+ 5715us) server_negotiation.cc:407] Connection header received
0809 00:46:45.328957 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:45.328965 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:45.329029 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:45.329168 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:45.337160 (+ 7992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:45.339006 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:45.340127 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:45.340751 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:45.340821 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:45.344265 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:45.344271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:45.344271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:45.344287 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:45.348067 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:45.348652 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:45.348656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:45.348657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:45.348793 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:45.349112 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:45.349113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:45.349114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:45.349462 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:45.349623 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:45.349936 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 00:46:45.350059 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":113,"threads_started":1}
I20260809 00:46:45.351882 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:45.362567 29542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:45.353677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:48253 (local address 127.0.0.1:36187)
0809 00:46:45.353721 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:45.353723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:45.353893 (+ 170us) server_negotiation.cc:407] Connection header received
0809 00:46:45.353955 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:45.353956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:45.353967 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:45.354023 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:45.354802 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:45.355741 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:45.357088 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:45.357458 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:45.357534 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:45.359248 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:45.359250 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:45.359250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:45.359258 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:45.361100 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:45.361607 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:45.361609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:45.361609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:45.361674 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:45.361961 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:45.361961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:45.361962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:45.362098 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:45.362219 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:45.362393 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 00:46:45.362465 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:46:45.365186 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 29430
W20260809 00:46:45.383209 29412 connection.cc:535] client connection to 127.0.0.1:36187 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36187): Connection reset by peer (error 104)
2026-08-09T00:46:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:45 dist-test-slave-c12x krb5kdc[29580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:45 dist-test-slave-c12x krb5kdc[29580](info): set up 2 sockets
Aug 09 00:46:45 dist-test-slave-c12x krb5kdc[29580](info): commencing operation
krb5kdc: starting...
W20260809 00:46:46.629288 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:46 dist-test-slave-c12x krb5kdc[29580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:46.762945 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:46.848949 29601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:46.849238 29601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:46.849282 29601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:46.850509 29601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:46.850585 29601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:46.850610 29601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:46.850632 29601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:46.850660 29601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:46.852340 29601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34001
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:46.852655 29601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:46.852906 29601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:46.858439 29614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:46.858556 29608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:46.858585 29601 server_base.cc:1034] running on GCE node
W20260809 00:46:46.858462 29610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:46.858893 29601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:46.859232 29601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:46.860441 29601 hybrid_clock.cc:648] HybridClock initialized: now 1786236406860413 us; error 50 us; skew 500 ppm
Aug 09 00:46:46 dist-test-slave-c12x krb5kdc[29580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236406, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:46.863832 29601 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:46.864374 29601 webserver.cc:460] Webserver started at http://127.0.0.1:41329/ using document root <none> and password file <none>
I20260809 00:46:46.864645 29601 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:46.864706 29601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:46.864861 29601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:46.866048 29601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "2f35b324605341009a0c7d48fdce6753"
format_stamp: "Formatted at 2026-08-09 00:46:46 on dist-test-slave-c12x"
server_key: "07b83357f9c30bdca06660a2fe4febe7"
server_key_iv: "351aa3b5673c541c7ca2d6fc758352f3"
server_key_version: "encryptionkey@0"
I20260809 00:46:46.866451 29601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "2f35b324605341009a0c7d48fdce6753"
format_stamp: "Formatted at 2026-08-09 00:46:46 on dist-test-slave-c12x"
server_key: "07b83357f9c30bdca06660a2fe4febe7"
server_key_iv: "351aa3b5673c541c7ca2d6fc758352f3"
server_key_version: "encryptionkey@0"
I20260809 00:46:46.868076 29601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:46.869074 29622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:46.869302 29601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:46.869424 29601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "2f35b324605341009a0c7d48fdce6753"
format_stamp: "Formatted at 2026-08-09 00:46:46 on dist-test-slave-c12x"
server_key: "07b83357f9c30bdca06660a2fe4febe7"
server_key_iv: "351aa3b5673c541c7ca2d6fc758352f3"
server_key_version: "encryptionkey@0"
I20260809 00:46:46.869531 29601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:46.881237 29601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:46.883960 29601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:46.884155 29601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:46.888706 29601 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34001
I20260809 00:46:46.888783 29705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34001 every 8 connection(s)
I20260809 00:46:46.889236 29601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:46.890233 29708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:46.892814 29708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753: Bootstrap starting.
I20260809 00:46:46.893460 29708 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:46.893775 29708 log.cc:824] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:46.894562 29708 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753: No bootstrap required, opened a new log
I20260809 00:46:46.896309 29708 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f35b324605341009a0c7d48fdce6753" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260809 00:46:46.896457 29708 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:46.896481 29708 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f35b324605341009a0c7d48fdce6753, State: Initialized, Role: FOLLOWER
I20260809 00:46:46.896688 29708 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f35b324605341009a0c7d48fdce6753" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260809 00:46:46.896780 29708 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:46.896831 29708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:46.896884 29708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:46.897482 29708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f35b324605341009a0c7d48fdce6753" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260809 00:46:46.897614 29708 leader_election.cc:302] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f35b324605341009a0c7d48fdce6753; no voters:
I20260809 00:46:46.897819 29708 leader_election.cc:288] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:46.897858 29713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:46.898053 29713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [term 1 LEADER]: Becoming Leader. State: Replica: 2f35b324605341009a0c7d48fdce6753, State: Running, Role: LEADER
I20260809 00:46:46.898106 29708 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:46.898165 29713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f35b324605341009a0c7d48fdce6753" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } }
I20260809 00:46:46.898726 29714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f35b324605341009a0c7d48fdce6753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f35b324605341009a0c7d48fdce6753" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } } }
I20260809 00:46:46.898826 29714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:46.899001 29715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f35b324605341009a0c7d48fdce6753. Latest consensus state: current_term: 1 leader_uuid: "2f35b324605341009a0c7d48fdce6753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f35b324605341009a0c7d48fdce6753" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34001 } } }
I20260809 00:46:46.899266 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 29601
I20260809 00:46:46.899401 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:46.899649 31139 external_mini_cluster.cc:1428] Setting key 2d92197dd3e921f68a4c4a88d465c1cd
I20260809 00:46:46.899075 29715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:46.899050 29727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:46.900766 29727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:46.902446 29727 catalog_manager.cc:1347] Generated new cluster ID: 3ae97d71bd2b47469eff4b5603ac76bd
I20260809 00:46:46.902513 29727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:46.906150 29737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:46.900150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:43259 (local address 127.0.0.1:34001)
0809 00:46:46.901353 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:46.901361 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:46.901391 (+ 30us) server_negotiation.cc:407] Connection header received
0809 00:46:46.901480 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:46.901486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:46.901541 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:46.901699 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:46.901925 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:46.903124 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:46.904042 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:46.904496 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:46.904619 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:46.905403 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:46.905407 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:46.905408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:46.905600 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:46.905633 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:46.905645 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:46.905744 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:46.905935 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 00:46:46.905964 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1157,"thread_start_us":72,"threads_started":1}
I20260809 00:46:46.907172 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:46.911720 29737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:46.908689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:33443 (local address 127.0.0.1:34001)
0809 00:46:46.908731 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:46.908733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:46.908874 (+ 141us) server_negotiation.cc:407] Connection header received
0809 00:46:46.908931 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:46.908932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:46.908942 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:46.909000 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:46.909174 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:46.909676 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:46.910404 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:46.910650 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:46.910728 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:46.911464 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:46.911466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:46.911466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:46.911471 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:46.911485 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:46.911489 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:46.911548 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:46.911625 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 00:46:46.911637 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 00:46:46.913400 29727 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:46.914119 29727 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:46.918202 29727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f35b324605341009a0c7d48fdce6753: Generated new TSK 0
I20260809 00:46:46.918412 29727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:46.922555 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 29601
W20260809 00:46:46.933547 29578 connection.cc:535] client connection to 127.0.0.1:34001 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34001): Connection reset by peer (error 104)
2026-08-09T00:46:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:46 dist-test-slave-c12x krb5kdc[29752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:46 dist-test-slave-c12x krb5kdc[29752](info): set up 2 sockets
Aug 09 00:46:46 dist-test-slave-c12x krb5kdc[29752](info): commencing operation
krb5kdc: starting...
W20260809 00:46:48.160885 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:48 dist-test-slave-c12x krb5kdc[29752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 00:46:48.294075 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:48.384785 29774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:48.385021 29774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:48.385047 29774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:48.386183 29774 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:48.386219 29774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:48.386233 29774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:48.386246 29774 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:48.386258 29774 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:48.387755 29774 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:48.387969 29774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:48.388152 29774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:48.393332 29782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:48.393343 29780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:48.393343 29785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:48.393954 29774 server_base.cc:1034] running on GCE node
I20260809 00:46:48.394224 29774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:48.394563 29774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:48.395740 29774 hybrid_clock.cc:648] HybridClock initialized: now 1786236408395717 us; error 43 us; skew 500 ppm
Aug 09 00:46:48 dist-test-slave-c12x krb5kdc[29752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236408, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:48.402148 29774 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 00:46:48.402696 29774 webserver.cc:460] Webserver started at http://127.0.0.1:45615/ using document root <none> and password file <none>
I20260809 00:46:48.402956 29774 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:48.403030 29774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:48.403190 29774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:48.404412 29774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c"
format_stamp: "Formatted at 2026-08-09 00:46:48 on dist-test-slave-c12x"
server_key: "06e6fcd889be661eb5ad9c96fbdcb333"
server_key_iv: "fa19aa45c3bb3e27d0028ecf004e3205"
server_key_version: "encryptionkey@0"
I20260809 00:46:48.404817 29774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c"
format_stamp: "Formatted at 2026-08-09 00:46:48 on dist-test-slave-c12x"
server_key: "06e6fcd889be661eb5ad9c96fbdcb333"
server_key_iv: "fa19aa45c3bb3e27d0028ecf004e3205"
server_key_version: "encryptionkey@0"
I20260809 00:46:48.406277 29774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 00:46:48.407256 29794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:48.407620 29774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:48.407752 29774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c"
format_stamp: "Formatted at 2026-08-09 00:46:48 on dist-test-slave-c12x"
server_key: "06e6fcd889be661eb5ad9c96fbdcb333"
server_key_iv: "fa19aa45c3bb3e27d0028ecf004e3205"
server_key_version: "encryptionkey@0"
I20260809 00:46:48.407840 29774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:48.411583 29774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:48.413983 29774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:48.414165 29774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:48.418566 29774 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41359
I20260809 00:46:48.418606 29862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41359 every 8 connection(s)
I20260809 00:46:48.418977 29774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:48.420102 29863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:48.420583 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 29774
I20260809 00:46:48.420727 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:48.420917 31139 external_mini_cluster.cc:1428] Setting key 2cccd6f2a3944c349f87b6bcd1f69919
I20260809 00:46:48.423353 29863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c: Bootstrap starting.
I20260809 00:46:48.423820 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:48.421501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:45745 (local address 127.0.0.1:41359)
0809 00:46:48.421704 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:48.421716 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:48.421754 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:46:48.422303 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:48.422313 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:48.422350 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:48.422515 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:48.423213 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:48.423217 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:48.423217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:48.423222 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:48.423269 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:48.423284 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:48.423361 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:48.423426 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 00:46:48.423611 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260809 00:46:48.424141 29863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:48.424490 29863 log.cc:824] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:48.424937 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:48.425330 29863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c: No bootstrap required, opened a new log
I20260809 00:46:48.427562 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:48.426464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.121:55301 (local address 127.0.0.1:41359)
0809 00:46:48.426503 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:48.426505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:48.426610 (+ 105us) server_negotiation.cc:407] Connection header received
0809 00:46:48.426674 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:48.426675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:48.426679 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:48.426729 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:48.427335 (+ 606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:48.427336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:48.427337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:48.427340 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:48.427353 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:48.427356 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:48.427395 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:48.427478 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:46:48.427490 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:46:48.427759 29863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41359 } }
I20260809 00:46:48.427944 29863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:48.427982 29863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b56a026ba324d2b89b3f74cbfaf6d1c, State: Initialized, Role: FOLLOWER
I20260809 00:46:48.428107 29863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41359 } }
I20260809 00:46:48.428225 29863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:48.428270 29863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:48.428321 29863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:48.428905 29863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41359 } }
I20260809 00:46:48.429025 29863 leader_election.cc:302] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b56a026ba324d2b89b3f74cbfaf6d1c; no voters:
I20260809 00:46:48.429222 29863 leader_election.cc:288] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:48.429307 29876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:48.429476 29876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [term 1 LEADER]: Becoming Leader. State: Replica: 5b56a026ba324d2b89b3f74cbfaf6d1c, State: Running, Role: LEADER
I20260809 00:46:48.429487 29863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:48.429590 29876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41359 } }
I20260809 00:46:48.430004 29878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b56a026ba324d2b89b3f74cbfaf6d1c. Latest consensus state: current_term: 1 leader_uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41359 } } }
I20260809 00:46:48.430145 29878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:48.429993 29877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b56a026ba324d2b89b3f74cbfaf6d1c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41359 } } }
I20260809 00:46:48.430503 29877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:48.430680 29892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:48.431149 29892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:48.433017 29892 catalog_manager.cc:1347] Generated new cluster ID: c992307deba146ff932734653e26aed0
I20260809 00:46:48.433075 29892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:48.458215 29892 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:48.458442 29892 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:48.461251 29892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b56a026ba324d2b89b3f74cbfaf6d1c: Generated new TSK 0
I20260809 00:46:48.461478 29892 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:48.470310 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 29774
2026-08-09T00:46:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:48 dist-test-slave-c12x krb5kdc[29909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:48 dist-test-slave-c12x krb5kdc[29909](info): set up 2 sockets
Aug 09 00:46:48 dist-test-slave-c12x krb5kdc[29909](info): commencing operation
krb5kdc: starting...
W20260809 00:46:49.698089 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:49 dist-test-slave-c12x krb5kdc[29909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:49.835136 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39161
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:39161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:49.922129 29930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:49.922403 29930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:49.922446 29930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:49.923689 29930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:49.923766 29930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:49.923791 29930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:49.923821 29930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:49.923851 29930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:49.925380 29930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:39161
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:39161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:49.925690 29930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:49.925935 29930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:49.931023 29944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:49.931171 29939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:49.931025 29940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:49.931236 29930 server_base.cc:1034] running on GCE node
I20260809 00:46:49.931629 29930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:49.931931 29930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:49.933084 29930 hybrid_clock.cc:648] HybridClock initialized: now 1786236409933063 us; error 42 us; skew 500 ppm
Aug 09 00:46:49 dist-test-slave-c12x krb5kdc[29909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236409, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:49.935720 29930 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:49.936199 29930 webserver.cc:460] Webserver started at http://127.30.104.254:45245/ using document root <none> and password file <none>
I20260809 00:46:49.936389 29930 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:49.936430 29930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:49.936520 29930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:49.937644 29930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "2f62ab0d968a4f7e8a717daeb5016d84"
format_stamp: "Formatted at 2026-08-09 00:46:49 on dist-test-slave-c12x"
server_key: "3c9905d4317d68befc9d3ab91a3f55f1"
server_key_iv: "7d6fe9de971cecc2d6044bd5c0bbba3c"
server_key_version: "encryptionkey@0"
I20260809 00:46:49.937961 29930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "2f62ab0d968a4f7e8a717daeb5016d84"
format_stamp: "Formatted at 2026-08-09 00:46:49 on dist-test-slave-c12x"
server_key: "3c9905d4317d68befc9d3ab91a3f55f1"
server_key_iv: "7d6fe9de971cecc2d6044bd5c0bbba3c"
server_key_version: "encryptionkey@0"
I20260809 00:46:49.939293 29930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:46:49.940120 29953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:49.940313 29930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:46:49.940390 29930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "2f62ab0d968a4f7e8a717daeb5016d84"
format_stamp: "Formatted at 2026-08-09 00:46:49 on dist-test-slave-c12x"
server_key: "3c9905d4317d68befc9d3ab91a3f55f1"
server_key_iv: "7d6fe9de971cecc2d6044bd5c0bbba3c"
server_key_version: "encryptionkey@0"
I20260809 00:46:49.940445 29930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:49.953495 29930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:49.955966 29930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:49.956084 29930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:49.960283 29930 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:39161
I20260809 00:46:49.960338 30029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:39161 every 8 connection(s)
I20260809 00:46:49.960692 29930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:49.961387 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 29930
I20260809 00:46:49.961519 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:49.961730 31139 external_mini_cluster.cc:1428] Setting key 16b32ffe1b574294d6b7109330157fdb
I20260809 00:46:49.961727 30031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:49.965095 30031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84: Bootstrap starting.
I20260809 00:46:49.965898 30031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:49.966209 30031 log.cc:824] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:49.967046 30031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84: No bootstrap required, opened a new log
Aug 09 00:46:49 dist-test-slave-c12x krb5kdc[29909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:49.969867 30031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f62ab0d968a4f7e8a717daeb5016d84" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39161 } }
I20260809 00:46:49.970016 30031 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:49.970041 30031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f62ab0d968a4f7e8a717daeb5016d84, State: Initialized, Role: FOLLOWER
I20260809 00:46:49.970192 30031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f62ab0d968a4f7e8a717daeb5016d84" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39161 } }
I20260809 00:46:49.970281 30031 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:49.970353 30031 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:49.970412 30031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:49.971263 30031 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f62ab0d968a4f7e8a717daeb5016d84" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39161 } }
I20260809 00:46:49.971381 30031 leader_election.cc:302] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f62ab0d968a4f7e8a717daeb5016d84; no voters:
I20260809 00:46:49.971592 30031 leader_election.cc:288] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:49.971673 30038 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:49.971868 30038 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [term 1 LEADER]: Becoming Leader. State: Replica: 2f62ab0d968a4f7e8a717daeb5016d84, State: Running, Role: LEADER
I20260809 00:46:49.971901 30031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:49.971976 30038 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f62ab0d968a4f7e8a717daeb5016d84" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39161 } }
I20260809 00:46:49.972442 30039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f62ab0d968a4f7e8a717daeb5016d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f62ab0d968a4f7e8a717daeb5016d84" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39161 } } }
I20260809 00:46:49.972529 30039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:49.972762 30054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:49.973369 30054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:49.973332 30041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f62ab0d968a4f7e8a717daeb5016d84. Latest consensus state: current_term: 1 leader_uuid: "2f62ab0d968a4f7e8a717daeb5016d84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f62ab0d968a4f7e8a717daeb5016d84" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 39161 } } }
I20260809 00:46:49.973449 30041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:49.974463 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:49.962408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57565 (local address 127.30.104.254:39161)
0809 00:46:49.962610 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:49.962617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:49.962642 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:46:49.963280 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:49.963288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:49.963339 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:49.963536 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:49.964328 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:49.965238 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:49.966359 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:49.966770 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:49.966856 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:49.969729 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:49.969733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:49.969733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:49.969744 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:49.971916 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:49.972516 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:49.972520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:49.972521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:49.972818 (+ 297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:49.973121 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:49.973122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:49.973123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:49.973457 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:49.973631 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:49.974146 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 00:46:49.974256 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260809 00:46:49.975603 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:49.980837 30054 catalog_manager.cc:1347] Generated new cluster ID: 4ad9233ad4954d1fb34662261052451a
I20260809 00:46:49.980909 30054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:49.985473 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:49.977096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33521 (local address 127.30.104.254:39161)
0809 00:46:49.977143 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:49.977144 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:49.977248 (+ 104us) server_negotiation.cc:407] Connection header received
0809 00:46:49.977277 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:49.977278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:49.977289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:49.977361 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:49.977869 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:49.979346 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:49.980357 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:49.980648 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:49.980718 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:49.982313 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:49.982314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:49.982314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:49.982323 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:49.984019 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:49.984508 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:49.984510 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:49.984510 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:49.984636 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:49.984899 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:49.984900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:49.984900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:49.985068 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:49.985181 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:49.985301 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 00:46:49.985374 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 00:46:49.994277 30054 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:49.994877 30054 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:49.998842 30054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f62ab0d968a4f7e8a717daeb5016d84: Generated new TSK 0
I20260809 00:46:49.999074 30054 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:50.004334 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 29930
W20260809 00:46:50.017023 29906 connection.cc:535] client connection to 127.30.104.254:39161 recv error: Network error: failed to read from TLS socket (remote: 127.30.104.254:39161): Connection reset by peer (error 104)
2026-08-09T00:46:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:50 dist-test-slave-c12x krb5kdc[30083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:50 dist-test-slave-c12x krb5kdc[30083](info): set up 2 sockets
Aug 09 00:46:50 dist-test-slave-c12x krb5kdc[30083](info): commencing operation
krb5kdc: starting...
W20260809 00:46:51.234750 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:51 dist-test-slave-c12x krb5kdc[30083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:51.370877 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:43839
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:43839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:51.456089 30101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:51.456346 30101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:51.456386 30101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:51.457535 30101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:51.457610 30101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:51.457636 30101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:51.457664 30101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:51.457695 30101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:51.459210 30101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:43839
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:43839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:51.459546 30101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:51.459789 30101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:51.464730 30111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:51.464777 30117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:51.464749 30113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:51.465133 30101 server_base.cc:1034] running on GCE node
I20260809 00:46:51.465353 30101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:51.465662 30101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:51.466806 30101 hybrid_clock.cc:648] HybridClock initialized: now 1786236411466794 us; error 37 us; skew 500 ppm
Aug 09 00:46:51 dist-test-slave-c12x krb5kdc[30083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236411, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:51.469416 30101 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:51.469884 30101 webserver.cc:460] Webserver started at http://127.30.104.254:34905/ using document root <none> and password file <none>
I20260809 00:46:51.470139 30101 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:51.470208 30101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:51.470359 30101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:51.471477 30101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "7c76fd668e74431188c1aff10589f3f8"
format_stamp: "Formatted at 2026-08-09 00:46:51 on dist-test-slave-c12x"
server_key: "d98e19fd097a297833a8f5d6e6d99f19"
server_key_iv: "6250eff7c598719d95acf2c6084d8dd8"
server_key_version: "encryptionkey@0"
I20260809 00:46:51.471884 30101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "7c76fd668e74431188c1aff10589f3f8"
format_stamp: "Formatted at 2026-08-09 00:46:51 on dist-test-slave-c12x"
server_key: "d98e19fd097a297833a8f5d6e6d99f19"
server_key_iv: "6250eff7c598719d95acf2c6084d8dd8"
server_key_version: "encryptionkey@0"
I20260809 00:46:51.473249 30101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:51.474113 30128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:51.474316 30101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:51.474426 30101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "7c76fd668e74431188c1aff10589f3f8"
format_stamp: "Formatted at 2026-08-09 00:46:51 on dist-test-slave-c12x"
server_key: "d98e19fd097a297833a8f5d6e6d99f19"
server_key_iv: "6250eff7c598719d95acf2c6084d8dd8"
server_key_version: "encryptionkey@0"
I20260809 00:46:51.474502 30101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:51.492110 30101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:51.494056 30101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:51.494230 30101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:51.498377 30101 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:43839
I20260809 00:46:51.498431 30196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:43839 every 8 connection(s)
I20260809 00:46:51.498824 30101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:51.499825 30197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:51.502162 30197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8: Bootstrap starting.
I20260809 00:46:51.502769 30197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:51.503059 30197 log.cc:824] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:51.503860 30197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8: No bootstrap required, opened a new log
I20260809 00:46:51.505460 30197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c76fd668e74431188c1aff10589f3f8" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43839 } }
I20260809 00:46:51.505607 30197 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:51.505631 30197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c76fd668e74431188c1aff10589f3f8, State: Initialized, Role: FOLLOWER
I20260809 00:46:51.505815 30197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c76fd668e74431188c1aff10589f3f8" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43839 } }
I20260809 00:46:51.505880 30197 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:51.505903 30197 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:51.505931 30197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:51.506834 30197 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c76fd668e74431188c1aff10589f3f8" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43839 } }
I20260809 00:46:51.506922 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 30101
I20260809 00:46:51.506932 30197 leader_election.cc:302] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c76fd668e74431188c1aff10589f3f8; no voters:
I20260809 00:46:51.507073 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:51.507143 30197 leader_election.cc:288] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:51.507172 30202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:51.507289 31139 external_mini_cluster.cc:1428] Setting key f3a433d7235003521982dffcccf3b533
I20260809 00:46:51.507341 30202 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [term 1 LEADER]: Becoming Leader. State: Replica: 7c76fd668e74431188c1aff10589f3f8, State: Running, Role: LEADER
I20260809 00:46:51.507472 30197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:51.507418 30202 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c76fd668e74431188c1aff10589f3f8" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43839 } }
I20260809 00:46:51.508046 30204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c76fd668e74431188c1aff10589f3f8. Latest consensus state: current_term: 1 leader_uuid: "7c76fd668e74431188c1aff10589f3f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c76fd668e74431188c1aff10589f3f8" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43839 } } }
I20260809 00:46:51.508054 30203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c76fd668e74431188c1aff10589f3f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c76fd668e74431188c1aff10589f3f8" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 43839 } } }
I20260809 00:46:51.508162 30204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:51.508167 30203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:51.508404 30224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:51.509003 30224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:51.510845 30224 catalog_manager.cc:1347] Generated new cluster ID: 6029c6b8a17e473094aeb925449e5560
I20260809 00:46:51.510901 30224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:51.513460 30215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:51.508006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56709 (local address 127.30.104.254:43839)
0809 00:46:51.508322 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:51.508331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:51.508415 (+ 84us) server_negotiation.cc:407] Connection header received
0809 00:46:51.508568 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:51.508578 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:51.508649 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:51.509082 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:51.509205 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:51.510489 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:51.511424 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:51.511929 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:51.512046 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:51.512836 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:51.512840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:51.512840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:51.513024 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:51.513055 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:51.513066 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:51.513161 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:51.513244 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 00:46:51.513275 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":77,"threads_started":1}
I20260809 00:46:51.514411 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:51.521818 30224 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:51.522550 30224 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:51.523342 30215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:51.516058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51843 (local address 127.30.104.254:43839)
0809 00:46:51.516117 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:51.516119 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:51.519707 (+ 3588us) server_negotiation.cc:407] Connection header received
0809 00:46:51.519806 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:51.519807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:51.519822 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:51.519920 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:51.520098 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:51.520659 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:51.521821 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:51.522137 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:51.522229 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:51.523062 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:51.523064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:51.523064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:51.523069 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:51.523084 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:51.523089 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:51.523148 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:51.523225 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 00:46:51.523239 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:46:51.526543 30224 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c76fd668e74431188c1aff10589f3f8: Generated new TSK 0
I20260809 00:46:51.526731 30224 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:51.530045 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 30101
W20260809 00:46:51.543257 30077 connection.cc:535] client connection to 127.30.104.254:43839 recv error: Network error: failed to read from TLS socket (remote: 127.30.104.254:43839): Connection reset by peer (error 104)
2026-08-09T00:46:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1525 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:51 dist-test-slave-c12x krb5kdc[30250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:51 dist-test-slave-c12x krb5kdc[30250](info): set up 2 sockets
Aug 09 00:46:51 dist-test-slave-c12x krb5kdc[30250](info): commencing operation
krb5kdc: starting...
W20260809 00:46:52.763727 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:52 dist-test-slave-c12x krb5kdc[30250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:52.918614 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:38703
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:38703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:53.003996 30270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:53.004254 30270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:53.004292 30270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:53.005463 30270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:53.005541 30270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:53.005569 30270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:53.005602 30270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:53.005626 30270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:53.007187 30270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:38703
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:38703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:53.007519 30270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:53.007768 30270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:53.012939 30279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:53.012951 30280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:53.012951 30286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:53.013376 30270 server_base.cc:1034] running on GCE node
I20260809 00:46:53.013633 30270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:53.013908 30270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:53.015058 30270 hybrid_clock.cc:648] HybridClock initialized: now 1786236413015047 us; error 31 us; skew 500 ppm
Aug 09 00:46:53 dist-test-slave-c12x krb5kdc[30250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236413, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:53.017714 30270 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:53.018254 30270 webserver.cc:460] Webserver started at http://127.30.104.254:43587/ using document root <none> and password file <none>
I20260809 00:46:53.018517 30270 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:53.018571 30270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:53.018752 30270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:53.019961 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "dd52e029d2d34067852cdb9dba0bbdf9"
format_stamp: "Formatted at 2026-08-09 00:46:53 on dist-test-slave-c12x"
server_key: "085a5bf80a1faa82322a9f989e546dcc"
server_key_iv: "07e79c9edddc4a3a20693b17ab21bec7"
server_key_version: "encryptionkey@0"
I20260809 00:46:53.020344 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "dd52e029d2d34067852cdb9dba0bbdf9"
format_stamp: "Formatted at 2026-08-09 00:46:53 on dist-test-slave-c12x"
server_key: "085a5bf80a1faa82322a9f989e546dcc"
server_key_iv: "07e79c9edddc4a3a20693b17ab21bec7"
server_key_version: "encryptionkey@0"
I20260809 00:46:53.021799 30270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 00:46:53.022580 30295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:53.022807 30270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:53.022908 30270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "dd52e029d2d34067852cdb9dba0bbdf9"
format_stamp: "Formatted at 2026-08-09 00:46:53 on dist-test-slave-c12x"
server_key: "085a5bf80a1faa82322a9f989e546dcc"
server_key_iv: "07e79c9edddc4a3a20693b17ab21bec7"
server_key_version: "encryptionkey@0"
I20260809 00:46:53.022984 30270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:53.026521 30270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:53.029024 30270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:53.029198 30270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:53.033708 30270 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:38703
I20260809 00:46:53.033735 30359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:38703 every 8 connection(s)
I20260809 00:46:53.034102 30270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:53.034976 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 30270
I20260809 00:46:53.035126 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:53.035324 30362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:53.035373 31139 external_mini_cluster.cc:1428] Setting key 227071d2203580a81800b5b2b47e47e6
I20260809 00:46:53.038177 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:53.036039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51911 (local address 127.30.104.254:38703)
0809 00:46:53.036261 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:53.036272 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:53.036314 (+ 42us) server_negotiation.cc:407] Connection header received
0809 00:46:53.036774 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:53.036784 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:53.036823 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:53.037005 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:53.037665 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:53.037669 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:53.037670 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:53.037676 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:53.037725 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:53.037742 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:53.037824 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:53.037914 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 00:46:53.037969 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260809 00:46:53.039269 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:53.039548 30362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9: Bootstrap starting.
I20260809 00:46:53.040230 30362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:53.040573 30362 log.cc:824] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:53.041515 30362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9: No bootstrap required, opened a new log
I20260809 00:46:53.042105 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:53.040866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54607 (local address 127.30.104.254:38703)
0809 00:46:53.040936 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:53.040939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:53.041098 (+ 159us) server_negotiation.cc:407] Connection header received
0809 00:46:53.041169 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:53.041169 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:53.041176 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:53.041239 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:53.041884 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:53.041886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:53.041886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:53.041891 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:53.041907 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:53.041911 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:53.041977 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:53.042019 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 00:46:53.042031 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 00:46:53.043892 30362 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd52e029d2d34067852cdb9dba0bbdf9" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38703 } }
I20260809 00:46:53.044111 30362 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:53.044155 30362 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd52e029d2d34067852cdb9dba0bbdf9, State: Initialized, Role: FOLLOWER
I20260809 00:46:53.044286 30362 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd52e029d2d34067852cdb9dba0bbdf9" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38703 } }
I20260809 00:46:53.044379 30362 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:53.044423 30362 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:53.044474 30362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:53.045069 30362 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd52e029d2d34067852cdb9dba0bbdf9" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38703 } }
I20260809 00:46:53.045161 30362 leader_election.cc:302] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd52e029d2d34067852cdb9dba0bbdf9; no voters:
I20260809 00:46:53.045363 30362 leader_election.cc:288] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:53.045397 30376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:53.045624 30376 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [term 1 LEADER]: Becoming Leader. State: Replica: dd52e029d2d34067852cdb9dba0bbdf9, State: Running, Role: LEADER
I20260809 00:46:53.045655 30362 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:53.045763 30376 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd52e029d2d34067852cdb9dba0bbdf9" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38703 } }
I20260809 00:46:53.046176 30377 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd52e029d2d34067852cdb9dba0bbdf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd52e029d2d34067852cdb9dba0bbdf9" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38703 } } }
I20260809 00:46:53.046196 30379 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd52e029d2d34067852cdb9dba0bbdf9. Latest consensus state: current_term: 1 leader_uuid: "dd52e029d2d34067852cdb9dba0bbdf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd52e029d2d34067852cdb9dba0bbdf9" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38703 } } }
I20260809 00:46:53.046511 30377 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:53.046527 30379 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:53.046981 30395 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:53.047618 30395 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:53.049381 30395 catalog_manager.cc:1347] Generated new cluster ID: 16cc836e93454f6fb5a765849797c6d6
I20260809 00:46:53.049443 30395 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:53.055652 30395 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:53.055846 30395 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:53.061347 30395 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd52e029d2d34067852cdb9dba0bbdf9: Generated new TSK 0
I20260809 00:46:53.061614 30395 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:53.064503 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 30270
2026-08-09T00:46:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1534 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:53 dist-test-slave-c12x krb5kdc[30409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:53 dist-test-slave-c12x krb5kdc[30409](info): set up 2 sockets
Aug 09 00:46:53 dist-test-slave-c12x krb5kdc[30409](info): commencing operation
krb5kdc: starting...
W20260809 00:46:54.297650 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:54 dist-test-slave-c12x krb5kdc[30409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:54.428423 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:38949
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:38949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:54.516911 30430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:54.517159 30430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:54.517187 30430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:54.518638 30430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:54.518714 30430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:54.518730 30430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:54.518744 30430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:54.518756 30430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:54.520552 30430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:38949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:38949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30430
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:54.520853 30430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:54.521114 30430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:54.526485 30439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:54.526489 30440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:54.526595 30430 server_base.cc:1034] running on GCE node
W20260809 00:46:54.526489 30443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:54.527033 30430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:54.527320 30430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:54.528514 30430 hybrid_clock.cc:648] HybridClock initialized: now 1786236414528502 us; error 37 us; skew 500 ppm
Aug 09 00:46:54 dist-test-slave-c12x krb5kdc[30409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236414, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:54.536652 30430 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:54.537241 30430 webserver.cc:460] Webserver started at http://127.30.104.254:42021/ using document root <none> and password file <none>
I20260809 00:46:54.537523 30430 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:54.537598 30430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:54.537925 30430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:54.539170 30430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "993e06fea326428faf96134d154e0a42"
format_stamp: "Formatted at 2026-08-09 00:46:54 on dist-test-slave-c12x"
server_key: "c17c0fd7f684e677cac381e5ad0238bc"
server_key_iv: "89561671e5914aefb17004b3c883fa8f"
server_key_version: "encryptionkey@0"
I20260809 00:46:54.539639 30430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "993e06fea326428faf96134d154e0a42"
format_stamp: "Formatted at 2026-08-09 00:46:54 on dist-test-slave-c12x"
server_key: "c17c0fd7f684e677cac381e5ad0238bc"
server_key_iv: "89561671e5914aefb17004b3c883fa8f"
server_key_version: "encryptionkey@0"
I20260809 00:46:54.541139 30430 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:54.542037 30452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:54.542315 30430 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:46:54.542510 30430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "993e06fea326428faf96134d154e0a42"
format_stamp: "Formatted at 2026-08-09 00:46:54 on dist-test-slave-c12x"
server_key: "c17c0fd7f684e677cac381e5ad0238bc"
server_key_iv: "89561671e5914aefb17004b3c883fa8f"
server_key_version: "encryptionkey@0"
I20260809 00:46:54.542601 30430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:54.554441 30430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:54.556941 30430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:54.557125 30430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:54.561798 30430 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:38949
I20260809 00:46:54.561837 30527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:38949 every 8 connection(s)
I20260809 00:46:54.562211 30430 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:54.563166 30529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:54.564877 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 30430
I20260809 00:46:54.565011 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:54.565213 31139 external_mini_cluster.cc:1428] Setting key eb5625fddcaecc5de0e9abcf87281296
I20260809 00:46:54.566035 30529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42: Bootstrap starting.
I20260809 00:46:54.566668 30529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:54.567008 30529 log.cc:824] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:54.567914 30529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42: No bootstrap required, opened a new log
I20260809 00:46:54.570361 30529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e06fea326428faf96134d154e0a42" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38949 } }
I20260809 00:46:54.570566 30529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:54.570616 30529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993e06fea326428faf96134d154e0a42, State: Initialized, Role: FOLLOWER
I20260809 00:46:54.570791 30529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e06fea326428faf96134d154e0a42" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38949 } }
I20260809 00:46:54.570907 30529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:54.570955 30529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:54.571033 30529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:54.571777 30529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e06fea326428faf96134d154e0a42" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38949 } }
I20260809 00:46:54.571913 30529 leader_election.cc:302] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993e06fea326428faf96134d154e0a42; no voters:
I20260809 00:46:54.572109 30529 leader_election.cc:288] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:54.572165 30537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:54.572335 30537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [term 1 LEADER]: Becoming Leader. State: Replica: 993e06fea326428faf96134d154e0a42, State: Running, Role: LEADER
I20260809 00:46:54.572441 30529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:54.572535 30537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e06fea326428faf96134d154e0a42" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38949 } }
I20260809 00:46:54.573020 30537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993e06fea326428faf96134d154e0a42. Latest consensus state: current_term: 1 leader_uuid: "993e06fea326428faf96134d154e0a42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e06fea326428faf96134d154e0a42" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38949 } } }
Aug 09 00:46:54 dist-test-slave-c12x krb5kdc[30409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:54.573127 30539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "993e06fea326428faf96134d154e0a42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e06fea326428faf96134d154e0a42" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 38949 } } }
I20260809 00:46:54.573171 30537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:54.573235 30539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:54.573603 30547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:54.574517 30547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:54.576666 30547 catalog_manager.cc:1347] Generated new cluster ID: c07e189d558347999f32cd486888c259
I20260809 00:46:54.576725 30547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:54.577910 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:54.565821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47555 (local address 127.30.104.254:38949)
0809 00:46:54.566023 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:54.566038 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:54.566064 (+ 26us) server_negotiation.cc:407] Connection header received
0809 00:46:54.566546 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:54.566553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:54.566604 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:54.566766 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:54.567631 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:54.569052 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:54.569895 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:54.570563 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:54.570701 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:54.573868 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:54.573873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:54.573873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:54.573886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:54.576067 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:54.576504 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:54.576507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:54.576508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:54.576628 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:54.576894 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:54.576895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:54.576895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:54.577208 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:54.577341 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:54.577528 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 00:46:54.577677 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260809 00:46:54.578931 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:54.589052 30547 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:54.589372 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:54.580779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60687 (local address 127.30.104.254:38949)
0809 00:46:54.580841 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:54.580843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:54.581070 (+ 227us) server_negotiation.cc:407] Connection header received
0809 00:46:54.581130 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:54.581131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:54.581143 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:54.581204 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:54.581940 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:54.582816 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:54.583909 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:54.584266 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:54.584340 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:54.585913 (+ 1573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:54.585914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:54.585914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:54.585922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:54.587881 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:54.588338 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:54.588340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:54.588340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:54.588418 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:54.588727 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:54.588728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:54.588729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:54.588876 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:54.589013 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:54.589181 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 00:46:54.589275 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 00:46:54.589879 30547 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:54.601387 30547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 993e06fea326428faf96134d154e0a42: Generated new TSK 0
I20260809 00:46:54.601677 30547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:54.606138 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 30430
2026-08-09T00:46:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1544 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:54 dist-test-slave-c12x krb5kdc[30572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:54 dist-test-slave-c12x krb5kdc[30572](info): set up 2 sockets
Aug 09 00:46:54 dist-test-slave-c12x krb5kdc[30572](info): commencing operation
krb5kdc: starting...
W20260809 00:46:55.849447 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:55 dist-test-slave-c12x krb5kdc[30572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:55.984488 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:34309
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:34309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:56.071604 30590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:56.071846 30590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:56.071873 30590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:56.073035 30590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:56.073082 30590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:56.073100 30590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:56.073115 30590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:56.073129 30590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:56.074643 30590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:34309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:34309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:56.074873 30590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:56.075067 30590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:56.080442 30599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:56.080610 30590 server_base.cc:1034] running on GCE node
W20260809 00:46:56.080456 30602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:56.080525 30600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:56.080943 30590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:56.081295 30590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:56.082445 30590 hybrid_clock.cc:648] HybridClock initialized: now 1786236416082421 us; error 40 us; skew 500 ppm
Aug 09 00:46:56 dist-test-slave-c12x krb5kdc[30572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236416, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:56.085187 30590 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:56.085667 30590 webserver.cc:460] Webserver started at http://127.30.104.254:43533/ using document root <none> and password file <none>
I20260809 00:46:56.085911 30590 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:56.085983 30590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:56.086145 30590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:56.087284 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "1c22a64438884f3bad2cb3ca2d09e168"
format_stamp: "Formatted at 2026-08-09 00:46:56 on dist-test-slave-c12x"
server_key: "fd4c7592aefc74a829c59fdce6377afc"
server_key_iv: "719aee7c469987d58edce408c9751c19"
server_key_version: "encryptionkey@0"
I20260809 00:46:56.087710 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "1c22a64438884f3bad2cb3ca2d09e168"
format_stamp: "Formatted at 2026-08-09 00:46:56 on dist-test-slave-c12x"
server_key: "fd4c7592aefc74a829c59fdce6377afc"
server_key_iv: "719aee7c469987d58edce408c9751c19"
server_key_version: "encryptionkey@0"
I20260809 00:46:56.089138 30590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:56.089916 30613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:56.090098 30590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:56.090214 30590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "1c22a64438884f3bad2cb3ca2d09e168"
format_stamp: "Formatted at 2026-08-09 00:46:56 on dist-test-slave-c12x"
server_key: "fd4c7592aefc74a829c59fdce6377afc"
server_key_iv: "719aee7c469987d58edce408c9751c19"
server_key_version: "encryptionkey@0"
I20260809 00:46:56.090294 30590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:56.110055 30590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:56.112505 30590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:56.112723 30590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:56.117267 30590 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:34309
I20260809 00:46:56.117344 30687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:34309 every 8 connection(s)
I20260809 00:46:56.117776 30590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:56.118722 30688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:56.120966 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 30590
I20260809 00:46:56.121114 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:56.121279 30688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168: Bootstrap starting.
I20260809 00:46:56.121304 31139 external_mini_cluster.cc:1428] Setting key d7665fb884d65e8203efb5f6cc1d50d6
I20260809 00:46:56.122017 30688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:56.122318 30688 log.cc:824] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:56.123278 30688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168: No bootstrap required, opened a new log
I20260809 00:46:56.125252 30688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c22a64438884f3bad2cb3ca2d09e168" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34309 } }
I20260809 00:46:56.125422 30688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:56.125491 30688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c22a64438884f3bad2cb3ca2d09e168, State: Initialized, Role: FOLLOWER
I20260809 00:46:56.125641 30688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c22a64438884f3bad2cb3ca2d09e168" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34309 } }
I20260809 00:46:56.125741 30688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:56.125774 30688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:56.125825 30688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:56.126453 30688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c22a64438884f3bad2cb3ca2d09e168" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34309 } }
I20260809 00:46:56.126581 30688 leader_election.cc:302] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c22a64438884f3bad2cb3ca2d09e168; no voters:
I20260809 00:46:56.126763 30688 leader_election.cc:288] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:56.126802 30695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:56.127012 30695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [term 1 LEADER]: Becoming Leader. State: Replica: 1c22a64438884f3bad2cb3ca2d09e168, State: Running, Role: LEADER
I20260809 00:46:56.127029 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:56.121864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59627 (local address 127.30.104.254:34309)
0809 00:46:56.122054 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:56.122062 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:56.122107 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:46:56.122621 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:56.122629 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:56.122680 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:56.122822 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:56.123021 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:56.124044 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:56.124874 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:56.125291 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:56.125409 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:56.126177 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:56.126181 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:56.126181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:56.126366 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:56.126398 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:56.126409 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:56.126483 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:56.126642 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 00:46:56.126722 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
I20260809 00:46:56.127066 30688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:56.127151 30695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c22a64438884f3bad2cb3ca2d09e168" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34309 } }
I20260809 00:46:56.127480 30695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c22a64438884f3bad2cb3ca2d09e168. Latest consensus state: current_term: 1 leader_uuid: "1c22a64438884f3bad2cb3ca2d09e168" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c22a64438884f3bad2cb3ca2d09e168" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34309 } } }
I20260809 00:46:56.127588 30695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:56.127492 30696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c22a64438884f3bad2cb3ca2d09e168" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c22a64438884f3bad2cb3ca2d09e168" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 34309 } } }
I20260809 00:46:56.127727 30696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:56.128067 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:56.128532 30707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:56.129053 30707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:56.130699 30707 catalog_manager.cc:1347] Generated new cluster ID: 935e50bce51142eaba4b0e1e3f066b97
I20260809 00:46:56.130748 30707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:56.133425 30691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:56.129667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36487 (local address 127.30.104.254:34309)
0809 00:46:56.130072 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:56.130073 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:56.130080 (+ 7us) server_negotiation.cc:407] Connection header received
0809 00:46:56.130096 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:56.130096 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:56.130104 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:56.130174 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:56.130340 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:56.131039 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:56.132110 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:56.132400 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:56.132469 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:56.133176 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:56.133178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:56.133178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:56.133182 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:56.133196 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:56.133200 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:56.133249 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:56.133329 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 00:46:56.133352 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390}
I20260809 00:46:56.144683 30707 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:56.145169 30707 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:56.154699 30707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c22a64438884f3bad2cb3ca2d09e168: Generated new TSK 0
I20260809 00:46:56.154917 30707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:56.157735 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 30590
2026-08-09T00:46:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:56 dist-test-slave-c12x krb5kdc[30726](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:56 dist-test-slave-c12x krb5kdc[30726](info): set up 2 sockets
Aug 09 00:46:56 dist-test-slave-c12x krb5kdc[30726](info): commencing operation
krb5kdc: starting...
W20260809 00:46:57.404477 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:57 dist-test-slave-c12x krb5kdc[30726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:57.543234 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37631
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:36269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:37631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:57.641775 30746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:57.642064 30746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:57.642112 30746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:57.643929 30746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:57.644011 30746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:57.644043 30746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:57.644085 30746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:57.644115 30746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:57.646790 30746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:36269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:37631
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:37631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:57.647156 30746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:57.647521 30746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:57.653440 30756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:57.653463 30759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:57.653450 30754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:57.654068 30746 server_base.cc:1034] running on GCE node
I20260809 00:46:57.654337 30746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:57.654675 30746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:57.655846 30746 hybrid_clock.cc:648] HybridClock initialized: now 1786236417655826 us; error 32 us; skew 500 ppm
Aug 09 00:46:57 dist-test-slave-c12x krb5kdc[30726](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236417, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:57.658638 30746 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:57.659137 30746 webserver.cc:460] Webserver started at http://127.30.104.254:39439/ using document root <none> and password file <none>
I20260809 00:46:57.659385 30746 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:57.659478 30746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:57.659631 30746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:57.660881 30746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "9cad72ff278b44ccb67f42eb6cfc322b"
format_stamp: "Formatted at 2026-08-09 00:46:57 on dist-test-slave-c12x"
server_key: "52a25965ab725440dd0283486f5ac459"
server_key_iv: "d25d0869e34948eecb6b1b979840cf87"
server_key_version: "encryptionkey@0"
I20260809 00:46:57.661305 30746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "9cad72ff278b44ccb67f42eb6cfc322b"
format_stamp: "Formatted at 2026-08-09 00:46:57 on dist-test-slave-c12x"
server_key: "52a25965ab725440dd0283486f5ac459"
server_key_iv: "d25d0869e34948eecb6b1b979840cf87"
server_key_version: "encryptionkey@0"
I20260809 00:46:57.662835 30746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:46:57.663831 30768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:57.664080 30746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:57.664204 30746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "9cad72ff278b44ccb67f42eb6cfc322b"
format_stamp: "Formatted at 2026-08-09 00:46:57 on dist-test-slave-c12x"
server_key: "52a25965ab725440dd0283486f5ac459"
server_key_iv: "d25d0869e34948eecb6b1b979840cf87"
server_key_version: "encryptionkey@0"
I20260809 00:46:57.664292 30746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:57.668042 30746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:57.670579 30746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:57.670746 30746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:57.675308 30746 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:37631
I20260809 00:46:57.675359 30842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:37631 every 8 connection(s)
I20260809 00:46:57.675782 30746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:57.676856 30844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:57.679219 30844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b: Bootstrap starting.
I20260809 00:46:57.680097 30844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:57.680373 30844 log.cc:824] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:57.680666 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 30746
I20260809 00:46:57.680788 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:57.680980 31139 external_mini_cluster.cc:1428] Setting key 7888734f81587e6af728a9624570ee73
I20260809 00:46:57.681248 30844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b: No bootstrap required, opened a new log
I20260809 00:46:57.683693 30844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cad72ff278b44ccb67f42eb6cfc322b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37631 } }
I20260809 00:46:57.683894 30844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 00:46:57.683969 30844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cad72ff278b44ccb67f42eb6cfc322b, State: Initialized, Role: FOLLOWER
I20260809 00:46:57.684155 30844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cad72ff278b44ccb67f42eb6cfc322b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37631 } }
I20260809 00:46:57.684273 30844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 00:46:57.684321 30844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 00:46:57.684355 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:57.681563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44237 (local address 127.30.104.254:37631)
0809 00:46:57.681789 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:57.681798 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:57.681833 (+ 35us) server_negotiation.cc:407] Connection header received
0809 00:46:57.682645 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:57.682653 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:57.682682 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:57.682843 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:57.683700 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:57.683704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:57.683704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:57.683710 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:57.683755 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:57.683765 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:57.683839 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:57.683918 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 00:46:57.683964 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260809 00:46:57.684372 30844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:57.685243 30844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cad72ff278b44ccb67f42eb6cfc322b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37631 } }
I20260809 00:46:57.685359 30844 leader_election.cc:302] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cad72ff278b44ccb67f42eb6cfc322b; no voters:
I20260809 00:46:57.685422 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:46:57.685534 30844 leader_election.cc:288] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 00:46:57.685563 30851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:57.685734 30851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [term 1 LEADER]: Becoming Leader. State: Replica: 9cad72ff278b44ccb67f42eb6cfc322b, State: Running, Role: LEADER
I20260809 00:46:57.685906 30844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:57.685895 30851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cad72ff278b44ccb67f42eb6cfc322b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37631 } }
I20260809 00:46:57.687969 30854 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cad72ff278b44ccb67f42eb6cfc322b. Latest consensus state: current_term: 1 leader_uuid: "9cad72ff278b44ccb67f42eb6cfc322b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cad72ff278b44ccb67f42eb6cfc322b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37631 } } }
I20260809 00:46:57.688153 30853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cad72ff278b44ccb67f42eb6cfc322b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cad72ff278b44ccb67f42eb6cfc322b" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 37631 } } }
I20260809 00:46:57.688444 30853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:57.688436 30854 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b [sys.catalog]: This master's current role is: LEADER
W20260809 00:46:57.688417 30871 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:46:57.688565 30871 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:46:57.691807 30876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:57.692162 30876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:57.692265 30850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:57.687050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37705 (local address 127.30.104.254:37631)
0809 00:46:57.687088 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:57.687090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:57.687216 (+ 126us) server_negotiation.cc:407] Connection header received
0809 00:46:57.687296 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:57.687297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:57.687303 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:57.687355 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:57.691938 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:57.691940 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:57.691941 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 00:46:57.691947 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:57.691964 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 00:46:57.691969 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:57.692050 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:57.692113 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 00:46:57.692131 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 00:46:57.693975 30876 catalog_manager.cc:1347] Generated new cluster ID: 516cee150b8046d9a5cc8e88c1aa6b24
I20260809 00:46:57.694043 30876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:57.701099 30876 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:57.701318 30876 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:57.710966 30876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cad72ff278b44ccb67f42eb6cfc322b: Generated new TSK 0
I20260809 00:46:57.711225 30876 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:46:57.713634 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 30746
2026-08-09T00:46:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1562 ms)
[----------] 18 tests from AuthTokenIssuingTest (27744 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:46:57 dist-test-slave-c12x krb5kdc[30884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:46:57 dist-test-slave-c12x krb5kdc[30884](info): set up 2 sockets
Aug 09 00:46:57 dist-test-slave-c12x krb5kdc[30884](info): commencing operation
krb5kdc: starting...
W20260809 00:46:58.955593 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:46:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:46:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:46:59.085994 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:46371
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:46371,127.30.104.253:40203,127.30.104.252:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:46:59.175043 30904 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:46:59.175321 30904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:59.175374 30904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:59.175418 30904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:59.176645 30904 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:59.176725 30904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:59.176753 30904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:59.176787 30904 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:59.176820 30904 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:59.178385 30904 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:46371,127.30.104.253:40203,127.30.104.252:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:46371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:59.178682 30904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:59.178929 30904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:59.184130 30914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:59.184181 30909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:59.184165 30911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:59.184473 30904 server_base.cc:1034] running on GCE node
I20260809 00:46:59.184782 30904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:59.185077 30904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:59.186264 30904 hybrid_clock.cc:648] HybridClock initialized: now 1786236419186225 us; error 53 us; skew 500 ppm
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:59.189136 30904 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:46:59.189606 30904 webserver.cc:460] Webserver started at http://127.30.104.254:40903/ using document root <none> and password file <none>
I20260809 00:46:59.189836 30904 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:59.189904 30904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:59.190056 30904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:59.191154 30904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "18128cc1f4b52349c47cb4e3f2c4e67e"
server_key_iv: "490b587349f4b23a45b8b65da3d3f1b3"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.191617 30904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "18128cc1f4b52349c47cb4e3f2c4e67e"
server_key_iv: "490b587349f4b23a45b8b65da3d3f1b3"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.193038 30904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:59.193799 30922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:59.193977 30904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 00:46:59.194094 30904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "18128cc1f4b52349c47cb4e3f2c4e67e"
server_key_iv: "490b587349f4b23a45b8b65da3d3f1b3"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.194176 30904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:59.227385 30904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:59.229893 30904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:59.230064 30904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:59.234452 30904 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:46371
I20260809 00:46:59.234561 30998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:46371 every 8 connection(s)
I20260809 00:46:59.234911 30904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:46:59.235908 30999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:59.237469 30999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:46:59.242471 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 30904
I20260809 00:46:59.242604 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:46:59.242802 31139 external_mini_cluster.cc:1428] Setting key 3238a6ebde9f0963ee569ec9d8eecc54
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:59.247776 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.238197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.30.104.254:46371)
0809 00:46:59.238347 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.238360 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.238439 (+ 79us) server_negotiation.cc:407] Connection header received
0809 00:46:59.238942 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.238944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.238961 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.239066 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.239647 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.240475 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.241315 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.241635 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.241717 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.244111 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.244121 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.244121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.244129 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.245941 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.246413 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.246417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.246417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.246508 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.246762 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.246763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.246763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.246995 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.247061 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.247331 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.247582 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":155,"server-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260809 00:46:59.247808 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.238135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:46371 (local address 127.0.0.1:46142)
0809 00:46:59.238301 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.238346 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.238810 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.239105 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.239107 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.239374 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.239558 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.239562 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.240562 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.240562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.241236 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.241238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.241711 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.241712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.241768 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.242289 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.242297 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.244017 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.246020 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.246027 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.246030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.246315 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.246575 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.246576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.246577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.246696 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.247065 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.247072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.247160 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.247261 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.247525 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260809 00:46:59.248610 30999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:46:59.248999 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.248851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:57928)
0809 00:46:59.248903 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.248962 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:46:59.249177 30928 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
I20260809 00:46:59.249686 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.249572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:57936)
0809 00:46:59.249621 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.249655 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:46:59.249833 30999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:40203: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.104.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab.
Entry for principal kudu/127.30.104.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab.
I20260809 00:46:59.271088 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.253:40203
--webserver_interface=127.30.104.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab
--principal=kudu/127.30.104.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:46371,127.30.104.253:40203,127.30.104.252:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:46:59.299063 30999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } attempt: 1
I20260809 00:46:59.299654 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.299495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:57944)
0809 00:46:59.299567 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.299618 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 00:46:59.300089 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.299997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:57952)
0809 00:46:59.300026 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.300059 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 00:46:59.300235 30999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:40203: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
W20260809 00:46:59.357613 31009 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:46:59.357821 31009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:59.357841 31009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:59.357859 31009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:59.359071 31009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:59.359118 31009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:59.359133 31009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:59.359145 31009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:59.359157 31009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:59.360700 31009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:46371,127.30.104.253:40203,127.30.104.252:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab
--principal=kudu/127.30.104.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.253:40203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.104.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:59.360965 31009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:59.361205 31009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:59.366362 31018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:59.366372 31020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:59.366372 31017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:59.366432 31009 server_base.cc:1034] running on GCE node
I20260809 00:46:59.366871 31009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:59.367187 31009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:59.368434 31009 hybrid_clock.cc:648] HybridClock initialized: now 1786236419368355 us; error 36 us; skew 500 ppm
I20260809 00:46:59.368420 30999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } attempt: 2
I20260809 00:46:59.368873 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.368703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:57968)
0809 00:46:59.368789 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.368829 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 00:46:59.369289 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.369181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:57970)
0809 00:46:59.369227 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.369259 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:46:59.369421 30999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:40203: Network error: Client connection negotiation failed: client connection to 127.30.104.253:40203: connect: Connection refused (error 111)
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:59.371284 31009 init.cc:377] Logged in from keytab as kudu/127.30.104.253@KRBTEST.COM (short username kudu)
I20260809 00:46:59.371807 31009 webserver.cc:460] Webserver started at http://127.30.104.253:34125/ using document root <none> and password file <none>
I20260809 00:46:59.372072 31009 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:59.372140 31009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:59.372292 31009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:59.373456 31009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/data/instance:
uuid: "fcc7c52db94f48508a09a1910129f4fd"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "4d980aa5b72eb20a91246538b83a92e5"
server_key_iv: "0572801d97010fdd3e0fe477acef485b"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.373821 31009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/wal/instance:
uuid: "fcc7c52db94f48508a09a1910129f4fd"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "4d980aa5b72eb20a91246538b83a92e5"
server_key_iv: "0572801d97010fdd3e0fe477acef485b"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.375207 31009 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 00:46:59.376139 31031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:59.376346 31009 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:46:59.376449 31009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/wal
uuid: "fcc7c52db94f48508a09a1910129f4fd"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "4d980aa5b72eb20a91246538b83a92e5"
server_key_iv: "0572801d97010fdd3e0fe477acef485b"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.376535 31009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:59.388540 31009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:59.390552 31009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:59.390723 31009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:59.395013 31009 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.253:40203
I20260809 00:46:59.395047 31108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.253:40203 every 8 connection(s)
I20260809 00:46:59.395418 31009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
I20260809 00:46:59.396363 31109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:59.397660 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31009
I20260809 00:46:59.397778 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-1/wal/instance
I20260809 00:46:59.398006 31139 external_mini_cluster.cc:1428] Setting key 67b2208f9d049820bb0e4f129210b8cf
I20260809 00:46:59.398406 31109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.104.252@KRBTEST.COM; defaulting to no policy
I20260809 00:46:59.408074 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.399277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.30.104.254:46371)
0809 00:46:59.399347 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.399349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.399570 (+ 221us) server_negotiation.cc:407] Connection header received
0809 00:46:59.400074 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.400076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.400092 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.400148 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.400797 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.401303 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.402291 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.402514 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.402593 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.404828 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.404830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.404831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.404840 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.406517 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.406976 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.406978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.406979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.407045 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.407292 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.407293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.407294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.407413 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.407532 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.407854 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.407977 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:46:59.408236 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.399238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:46371 (local address 127.0.0.1:46154)
0809 00:46:59.399412 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.399480 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.399947 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.400216 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.400224 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.400492 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.400690 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.400695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.401403 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.401404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.402196 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.402198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.402601 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.402603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.402733 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.403303 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.403311 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.404730 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.406611 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.406618 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.406620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.406903 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.407123 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.407124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.407124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.407231 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.407537 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.407550 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.407698 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.407822 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.407997 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":96,"threads_started":1}
I20260809 00:46:59.408902 31109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.252@KRBTEST.COM" created.
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:46:59.417086 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.409243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:57986)
0809 00:46:59.409295 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.409311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.409346 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.409666 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.409667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.409837 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.409904 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.409906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.410800 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.410802 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.411322 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.411323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.411861 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.411862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.411927 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.412367 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.412372 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.413569 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.415902 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.415903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.415904 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.416101 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.416353 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.416353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.416353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.416423 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.416808 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.416809 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.416854 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.416927 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.417013 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:46:59.417237 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.409317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57986 (local address 127.30.104.253:40203)
0809 00:46:59.409498 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.409503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.409528 (+ 25us) server_negotiation.cc:407] Connection header received
0809 00:46:59.409566 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.409568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.409578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.409638 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.409992 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.410709 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.411406 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.411770 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.411862 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.413660 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.413670 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.413671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.413679 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.415783 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.416188 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.416192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.416193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.416267 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.416499 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.416500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.416500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.416721 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.416815 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.416986 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.417078 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":104,"threads_started":1}
I20260809 00:46:59.417898 31109 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:46:59.418269 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.418127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51740)
0809 00:46:59.418182 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.418239 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:46:59.418422 31038 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
I20260809 00:46:59.418927 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.418811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51750)
0809 00:46:59.418865 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.418898 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 00:46:59.419095 31109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:32811: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab.
Entry for principal kudu/127.30.104.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab.
I20260809 00:46:59.427115 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.252:32811
--webserver_interface=127.30.104.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab
--principal=kudu/127.30.104.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:46371,127.30.104.253:40203,127.30.104.252:32811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:46:59.460592 30999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } attempt: 3
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:46:59.468331 31109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } attempt: 1
I20260809 00:46:59.468644 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.461126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57996 (local address 127.30.104.253:40203)
0809 00:46:59.461197 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.461199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.461228 (+ 29us) server_negotiation.cc:407] Connection header received
0809 00:46:59.461258 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.461259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.461277 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.461337 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.461740 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.462283 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.463042 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.463270 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.463347 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.465652 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.465654 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.465654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.465664 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.467252 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.467731 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.467737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.467738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.467794 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.468026 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.468027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.468027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.468177 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.468256 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.468442 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.468537 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 00:46:59.468647 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.461038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:57996)
0809 00:46:59.461109 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.461120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.461186 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.461345 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.461346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.461549 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.461652 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.461655 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.462376 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.462377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.462952 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.462954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.463344 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.463346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.463476 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.464000 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.464005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.465536 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.467350 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.467351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.467352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.467639 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.467870 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.467871 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.467871 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.467951 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.468261 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.468262 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.468327 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.468413 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.468536 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 00:46:59.468781 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.468672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51754)
0809 00:46:59.468726 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.468753 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 00:46:59.469056 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.468990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51756)
0809 00:46:59.469019 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.469038 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 00:46:59.469141 31109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:32811: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
I20260809 00:46:59.469084 30999 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:46:59.469452 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.469333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51768)
0809 00:46:59.469393 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.469424 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:46:59.469770 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.469687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51776)
0809 00:46:59.469730 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.469755 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:46:59.469873 30999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:32811: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
I20260809 00:46:59.501026 30999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } attempt: 1
I20260809 00:46:59.501503 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.501365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51784)
0809 00:46:59.501441 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.501472 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 00:46:59.501852 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.501760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51790)
0809 00:46:59.501804 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.501834 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:46:59.501977 30999 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:32811: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
W20260809 00:46:59.516115 31120 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:46:59.516386 31120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:46:59.516418 31120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:46:59.516462 31120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:46:59.517689 31120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:46:59.517760 31120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:46:59.517786 31120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:46:59.517820 31120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:46:59.517846 31120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:46:59.519531 31120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:46371,127.30.104.253:40203,127.30.104.252:32811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab
--principal=kudu/127.30.104.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.252:32811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.104.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:46:59.519824 31120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:46:59.520069 31120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:46:59.525404 31131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:59.525417 31133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:46:59.525404 31130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:46:59.525609 31120 server_base.cc:1034] running on GCE node
I20260809 00:46:59.525904 31120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:46:59.526202 31120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:46:59.527357 31120 hybrid_clock.cc:648] HybridClock initialized: now 1786236419527344 us; error 38 us; skew 500 ppm
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:46:59.530136 31120 init.cc:377] Logged in from keytab as kudu/127.30.104.252@KRBTEST.COM (short username kudu)
I20260809 00:46:59.530620 31120 webserver.cc:460] Webserver started at http://127.30.104.252:33409/ using document root <none> and password file <none>
I20260809 00:46:59.530875 31120 fs_manager.cc:359] Metadata directory not provided
I20260809 00:46:59.530951 31120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:46:59.531097 31120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:46:59.532238 31120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/data/instance:
uuid: "777a8fa797b44660a133826e5582dc92"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "41cfbfd475a4d4f8fc80482533b70b29"
server_key_iv: "4fc95c50d1cb28b53bab32bc6c47fd31"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.532609 31120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/wal/instance:
uuid: "777a8fa797b44660a133826e5582dc92"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "41cfbfd475a4d4f8fc80482533b70b29"
server_key_iv: "4fc95c50d1cb28b53bab32bc6c47fd31"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.534046 31120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 00:46:59.534941 31151 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:46:59.535161 31120 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 00:46:59.535276 31120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/wal
uuid: "777a8fa797b44660a133826e5582dc92"
format_stamp: "Formatted at 2026-08-09 00:46:59 on dist-test-slave-c12x"
server_key: "41cfbfd475a4d4f8fc80482533b70b29"
server_key_iv: "4fc95c50d1cb28b53bab32bc6c47fd31"
server_key_version: "encryptionkey@0"
I20260809 00:46:59.535353 31120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:46:59.537282 31109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } attempt: 2
I20260809 00:46:59.537746 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.537603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51798)
0809 00:46:59.537669 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.537711 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 00:46:59.538197 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.538055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51800)
0809 00:46:59.538126 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.538172 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 00:46:59.538415 31109 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:32811: Network error: Client connection negotiation failed: client connection to 127.30.104.252:32811: connect: Connection refused (error 111)
I20260809 00:46:59.552314 31120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:46:59.554739 31120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:46:59.554908 31120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:46:59.559368 31120 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.252:32811
I20260809 00:46:59.559427 31218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.252:32811 every 8 connection(s)
I20260809 00:46:59.559824 31120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
I20260809 00:46:59.560814 31219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:46:59.562347 31219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:46:59.563246 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31120
I20260809 00:46:59.563359 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786236131192310-31139-0/minicluster-data/master-2/wal/instance
I20260809 00:46:59.563648 31139 external_mini_cluster.cc:1428] Setting key 6be595fe5f8efed2d6aa620f199d2103
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:46:59.572113 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.563113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46170 (local address 127.30.104.254:46371)
0809 00:46:59.563190 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.563192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.563390 (+ 198us) server_negotiation.cc:407] Connection header received
0809 00:46:59.564146 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.564147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.564159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.564214 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.564847 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.565337 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.566239 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.566438 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.566515 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.569060 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.569062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.569062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.569069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.570648 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.571070 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.571071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.571072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.571130 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.571389 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.571390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.571390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.571567 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.571642 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.571927 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.572015 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 00:46:59.572296 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.563045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:46371 (local address 127.0.0.1:46170)
0809 00:46:59.563248 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.563300 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.564025 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.564302 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.564306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.564575 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.564754 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.564758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.565429 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.565430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.566148 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.566150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.566520 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.566521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.566657 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.567211 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.567221 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.568949 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.570737 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.570744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.570746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.570993 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.571202 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.571203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.571203 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.571324 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.571635 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.571642 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.571786 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.571891 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.572055 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":91,"threads_started":1}
I20260809 00:46:59.572973 31219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:46:59.573133 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.564056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51802 (local address 127.30.104.252:32811)
0809 00:46:59.564211 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.564221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.564376 (+ 155us) server_negotiation.cc:407] Connection header received
0809 00:46:59.564439 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.564442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.564467 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.564574 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.565194 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.565971 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.566728 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.567174 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.567269 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.569953 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.569961 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.569961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.569970 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.571620 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.572002 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.572005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.572005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.572071 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.572281 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.572282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.572282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.572599 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.572770 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.572881 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.572962 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:46:59.577106 30999 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } attempt: 2
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:46:59.580248 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.573376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58002 (local address 127.30.104.253:40203)
0809 00:46:59.573418 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.573420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.573428 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:46:59.573453 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.573453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.573468 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.573514 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.573850 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.574446 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.575107 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.575319 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.575394 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.577319 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.577321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.577322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.577330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.578865 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.579239 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.579241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.579241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.579302 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.579692 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.579693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.579693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.579802 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.579873 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.580075 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.580160 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 00:46:59.580545 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.573274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:40203 (local address 127.0.0.1:58002)
0809 00:46:59.573311 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.573322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.573362 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.573534 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.573535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.573699 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.573764 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.573766 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.574529 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.574530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.575015 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.575017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.575390 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.575391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.575495 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.575919 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.575937 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.577246 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.578952 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.578953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.578953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.579181 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.579377 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.579378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.579379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.579468 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.579879 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.579880 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.579925 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.580350 (+ 425us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.580458 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 00:46:59.581214 31219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:46:59.582165 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.573921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46178 (local address 127.30.104.254:46371)
0809 00:46:59.573962 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.573964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.574009 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:46:59.574068 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.574069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.574078 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.574130 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.574558 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.575006 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.575798 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.576110 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.576183 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.579166 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.579167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.579168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.579174 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.580898 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.581297 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.581298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.581298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.581361 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.581579 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.581581 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.581581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.581709 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.581937 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.581976 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.582049 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:46:59.585546 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.577369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51816)
0809 00:46:59.577417 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.577467 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.577517 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.577831 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.577832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.578022 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.578064 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.578066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.579031 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.579032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.579915 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.579918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.580514 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.580515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.580573 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.581081 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.581086 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.582626 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.584358 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.584359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.584360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.584685 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.584898 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.584899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.584900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.584962 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.585207 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.585208 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.585254 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.585333 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.585450 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 00:46:59.585544 31223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.577433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51816 (local address 127.30.104.252:32811)
0809 00:46:59.577467 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.577469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.577503 (+ 34us) server_negotiation.cc:407] Connection header received
0809 00:46:59.577738 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.577739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.577749 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.577811 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.578150 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.578961 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.579991 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.580405 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.580713 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.582707 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.582709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.582709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.582717 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.584272 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.584773 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.584774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.584775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.584840 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.585032 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.585033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.585033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.585141 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.585214 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.585371 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.585446 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:46:59.587694 30999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5: Bootstrap starting.
I20260809 00:46:59.588544 30999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:59.588907 30999 log.cc:824] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5: Log is configured to *not* fsync() on all Append() calls
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:46:59.589844 30999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5: No bootstrap required, opened a new log
I20260809 00:46:59.590858 31221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.581504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51820)
0809 00:46:59.581856 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.581873 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.581926 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.582642 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.582643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.582839 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.582878 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.582879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.583883 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.583884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.584715 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.584718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.585630 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.585631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.585679 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.586087 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.586092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.587665 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.589342 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.589342 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.589343 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.589552 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.589736 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.589737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.589737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.589798 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.590176 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.590178 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.590227 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.590311 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.590723 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335}
I20260809 00:46:59.590973 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.581568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51820 (local address 127.30.104.252:32811)
0809 00:46:59.582462 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.582464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.582472 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:46:59.582492 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.582493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.582502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.582641 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.582960 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.583791 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.584801 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.585513 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.585581 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.587758 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.587759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.587760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.587767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.589246 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.589607 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.589608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.589608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.589662 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.589860 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.589861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.589862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.590099 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.590182 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.590350 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.590800 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":349,"server-negotiator.queue_time_us":871,"thread_start_us":227,"threads_started":1}
I20260809 00:46:59.592247 30999 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } }
I20260809 00:46:59.592461 30999 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d0bf3035e0a4cfdaf61e238f78a72f5, State: Initialized, Role: FOLLOWER
I20260809 00:46:59.592631 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.583251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.30.104.253:40203)
0809 00:46:59.583308 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.583310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.583358 (+ 48us) server_negotiation.cc:407] Connection header received
0809 00:46:59.583461 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.583462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.583473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.583525 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.584220 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.584745 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.585777 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.586129 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.586176 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.589557 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.589559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.589559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.589568 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.591354 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.591782 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.591783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.591783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.591845 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.592076 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.592077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.592077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.592214 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.592392 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.592446 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.592527 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:46:59.592582 30999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } }
I20260809 00:46:59.592834 31219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92: Bootstrap starting.
I20260809 00:46:59.592906 31229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.592981 30999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:59.593017 31229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:46:59.593633 31219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:59.594027 31219 log.cc:824] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:59.594689 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 00:46:59.595043 31248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:46:59.595094 31248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:46:59.595767 30959 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46178
I20260809 00:46:59.595847 31219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92: No bootstrap required, opened a new log
I20260809 00:46:59.595875 30959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 00:46:59.595927 30959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:59.596947 30959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } }
I20260809 00:46:59.597167 30959 leader_election.cc:288] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [CANDIDATE]: Term 1 election: Requested vote from peers fcc7c52db94f48508a09a1910129f4fd (127.30.104.253:40203), 777a8fa797b44660a133826e5582dc92 (127.30.104.252:32811)
I20260809 00:46:59.597911 31219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } }
I20260809 00:46:59.598055 31219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 777a8fa797b44660a133826e5582dc92, State: Initialized, Role: FOLLOWER
I20260809 00:46:59.598133 31219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } }
I20260809 00:46:59.598443 31219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:59.598402 31255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.598508 31255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:46:59.595983 30904 master_runner.cc:186] Error getting master registration for 127.30.104.253:40203: OK, instance_id { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" instance_seqno: 1786236419393833 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:46:59.598963 30904 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:46:59.599308 31187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "777a8fa797b44660a133826e5582dc92"
I20260809 00:46:59.599426 31187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:59.600874 31187 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0bf3035e0a4cfdaf61e238f78a72f5 in term 1.
I20260809 00:46:59.601572 30928 leader_election.cc:302] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d0bf3035e0a4cfdaf61e238f78a72f5, 777a8fa797b44660a133826e5582dc92; no voters:
I20260809 00:46:59.601838 31229 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:46:59.602129 31229 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [term 1 LEADER]: Becoming Leader. State: Replica: 3d0bf3035e0a4cfdaf61e238f78a72f5, State: Running, Role: LEADER
I20260809 00:46:59.602435 31229 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } }
I20260809 00:46:59.603128 31229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d0bf3035e0a4cfdaf61e238f78a72f5. Latest consensus state: current_term: 1 leader_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.603245 31229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:59.604722 31120 master_runner.cc:186] Error getting master registration for 127.30.104.253:40203: OK, instance_id { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" instance_seqno: 1786236419393833 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:46:59.607101 31120 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 00:46:59.610374 31282 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:46:59.610504 31282 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:46:59.612208 31285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:46:59.612520 31285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:46:59.614408 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.600484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.30.104.253:40203)
0809 00:46:59.600544 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.600546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.601101 (+ 555us) server_negotiation.cc:407] Connection header received
0809 00:46:59.601310 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.601311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.601331 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.601376 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.604576 (+ 3200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.605280 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.606286 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.606625 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.606688 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.609156 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.609157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.609157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.609166 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.611003 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.613302 (+ 2299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.613305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.613305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.613393 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.613642 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.613643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.613644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.613796 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.613891 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.614227 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.614317 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:46:59.615388 31005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.599950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46188 (local address 127.30.104.254:46371)
0809 00:46:59.599999 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.600001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.600783 (+ 782us) server_negotiation.cc:407] Connection header received
0809 00:46:59.600946 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.600948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.600959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.601002 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.602415 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.603281 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.604416 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.604713 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.604777 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.609365 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.609366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.609367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.609374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.611737 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.613349 (+ 1612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.613351 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.613351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.613418 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.614759 (+ 1341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.614760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.614761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.614900 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.614972 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.615202 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.615292 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:46:59.616771 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.601139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51830 (local address 127.30.104.252:32811)
0809 00:46:59.601219 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.601220 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.606420 (+ 5200us) server_negotiation.cc:407] Connection header received
0809 00:46:59.606513 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.606514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.606526 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.606556 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.607390 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.608148 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.609327 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.609657 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.609719 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.612748 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.612750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.612751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.612760 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.614523 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.615416 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.615418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.615418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.615506 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.615833 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.615834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.615834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.616049 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.616207 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.616503 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.616653 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 00:46:59.620549 31187 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [term 1 FOLLOWER]: Refusing update from remote peer 3d0bf3035e0a4cfdaf61e238f78a72f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:59.620981 31283 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:59.622833 31255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d0bf3035e0a4cfdaf61e238f78a72f5. Latest consensus state: current_term: 1 leader_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.622953 31255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:46:59.629594 31109 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } attempt: 3
Aug 09 00:46:59 dist-test-slave-c12x krb5kdc[30884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236419, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:46:59.639420 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.629940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:32811 (local address 127.0.0.1:51840)
0809 00:46:59.629995 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:46:59.630012 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:46:59.630056 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:46:59.630236 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:46:59.630237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:46:59.630465 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:46:59.630523 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.630526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.631490 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.631491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.632271 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:46:59.632273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.632816 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.632817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:46:59.632882 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.633422 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:46:59.633427 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:46:59.636202 (+ 2775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:46:59.638255 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.638256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.638256 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.638494 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.638751 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:46:59.638752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:46:59.638752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:46:59.638810 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:46:59.639088 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:46:59.639089 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:46:59.639135 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 00:46:59.639196 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:46:59.639308 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 00:46:59.639882 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:46:59.630044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51840 (local address 127.30.104.252:32811)
0809 00:46:59.630105 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.630107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.630117 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:46:59.630151 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.630152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.630162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.630229 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.630602 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.631355 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.632348 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:46:59.632684 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:46:59.632752 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:46:59.636315 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:46:59.636316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:46:59.636317 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:46:59.636325 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:46:59.638145 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.638623 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.638624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.638624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.638684 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:46:59.638872 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:46:59.638873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:46:59.638873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:46:59.639006 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:46:59.639063 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:46:59.639716 (+ 653us) server_negotiation.cc:299] Negotiation successful
0809 00:46:59.639798 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 00:46:59.641490 31109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd: Bootstrap starting.
I20260809 00:46:59.642261 31109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd: Neither blocks nor log segments found. Creating new log.
I20260809 00:46:59.642582 31109 log.cc:824] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd: Log is configured to *not* fsync() on all Append() calls
I20260809 00:46:59.643993 31255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.644102 31255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:46:59.643393 31283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.644361 31283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:59.645071 31285 catalog_manager.cc:1347] Generated new cluster ID: c9c4084cdd4f44afaaa241e3b145a07d
I20260809 00:46:59.645152 31285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:46:59.645081 31287 mvcc.cc:205] Tried to move back new op lower bound from 7316424374739103744 to 7316424374693072896. Current Snapshot: MvccSnapshot[applied={T|T < 7316424374739103744 or (T in {7316424374739103744})}]
I20260809 00:46:59.652295 31109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd: No bootstrap required, opened a new log
I20260809 00:46:59.653175 31285 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:46:59.654116 31285 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:46:59.654681 31109 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } }
I20260809 00:46:59.654845 31109 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc7c52db94f48508a09a1910129f4fd, State: Initialized, Role: FOLLOWER
I20260809 00:46:59.654935 31109 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } }
I20260809 00:46:59.655319 31109 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:46:59.656219 31068 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:46:59.657294 31068 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [term 1 FOLLOWER]: Refusing update from remote peer 3d0bf3035e0a4cfdaf61e238f78a72f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:46:59.657729 31229 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:46:59.658212 31069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fcc7c52db94f48508a09a1910129f4fd"
I20260809 00:46:59.659121 31069 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d0bf3035e0a4cfdaf61e238f78a72f5 in term 1.
I20260809 00:46:59.660975 31296 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.661149 31296 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:46:59.661247 31296 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d0bf3035e0a4cfdaf61e238f78a72f5. Latest consensus state: current_term: 1 leader_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.661357 31296 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:46:59.661427 31296 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.661513 31296 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:46:59.664075 31314 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:46:59.664242 31314 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:46:59.665364 31317 mvcc.cc:205] Tried to move back new op lower bound from 7316424374893858816 to 7316424374693072896. Current Snapshot: MvccSnapshot[applied={T|T < 7316424374739103744}]
I20260809 00:46:59.665814 31229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d0bf3035e0a4cfdaf61e238f78a72f5" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 46371 } } peers { permanent_uuid: "fcc7c52db94f48508a09a1910129f4fd" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 40203 } } peers { permanent_uuid: "777a8fa797b44660a133826e5582dc92" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 32811 } } }
I20260809 00:46:59.665913 31229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5 [sys.catalog]: This master's current role is: LEADER
I20260809 00:46:59.666260 31285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d0bf3035e0a4cfdaf61e238f78a72f5: Generated new TSK 0
I20260809 00:46:59.666832 31285 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 00:46:59.700847 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.699176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58026 (local address 127.30.104.253:40203)
0809 00:46:59.699225 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.699227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.700132 (+ 905us) server_negotiation.cc:407] Connection header received
0809 00:46:59.700243 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.700245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.700257 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.700289 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.700772 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58026: BlockingRecv error: recv got EOF from 127.0.0.1:58026 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 00:46:59.701861 31226 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.699622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51850 (local address 127.30.104.252:32811)
0809 00:46:59.699684 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.699686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.701101 (+ 1415us) server_negotiation.cc:407] Connection header received
0809 00:46:59.701292 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.701294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.701305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.701337 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.701797 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51850: BlockingRecv error: recv got EOF from 127.0.0.1:51850 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:46:59.702474 31005 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.699840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.30.104.254:46371)
0809 00:46:59.699883 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.699885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.701430 (+ 1545us) server_negotiation.cc:407] Connection header received
0809 00:46:59.702079 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.702080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.702091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.702122 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.702418 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46190: BlockingRecv error: recv got EOF from 127.0.0.1:46190 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:46:59.705147 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.704270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.30.104.253:40203)
0809 00:46:59.704308 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.704310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.704594 (+ 284us) server_negotiation.cc:407] Connection header received
0809 00:46:59.704710 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.704711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.704732 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.704763 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.705093 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58028: BlockingRecv error: recv got EOF from 127.0.0.1:58028 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:46:59.708364 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.707521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58040 (local address 127.30.104.253:40203)
0809 00:46:59.707568 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.707570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.707837 (+ 267us) server_negotiation.cc:407] Connection header received
0809 00:46:59.707939 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.707940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.707952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.707981 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.708316 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58040: BlockingRecv error: recv got EOF from 127.0.0.1:58040 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 00:46:59.712419 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.711554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58048 (local address 127.30.104.253:40203)
0809 00:46:59.711597 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.711599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.711871 (+ 272us) server_negotiation.cc:407] Connection header received
0809 00:46:59.711968 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.711970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.711982 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.712011 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.712361 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58048: BlockingRecv error: recv got EOF from 127.0.0.1:58048 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 00:46:59.718633 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.717677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.30.104.253:40203)
0809 00:46:59.717723 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.717726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.718015 (+ 289us) server_negotiation.cc:407] Connection header received
0809 00:46:59.718127 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.718128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.718140 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.718172 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.718575 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58058: BlockingRecv error: recv got EOF from 127.0.0.1:58058 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:46:59.729247 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.728094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.30.104.253:40203)
0809 00:46:59.728147 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.728149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.728507 (+ 358us) server_negotiation.cc:407] Connection header received
0809 00:46:59.728636 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.728638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.728652 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.728689 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.729164 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58072: BlockingRecv error: recv got EOF from 127.0.0.1:58072 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 00:46:59.747961 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.746770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.30.104.253:40203)
0809 00:46:59.746820 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.746823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.747182 (+ 359us) server_negotiation.cc:407] Connection header received
0809 00:46:59.747310 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.747311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.747325 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.747398 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.747896 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58086: BlockingRecv error: recv got EOF from 127.0.0.1:58086 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 00:46:59.782456 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.781389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.30.104.253:40203)
0809 00:46:59.781439 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.781441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.781765 (+ 324us) server_negotiation.cc:407] Connection header received
0809 00:46:59.781882 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.781883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.781897 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.781933 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.782334 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58100: BlockingRecv error: recv got EOF from 127.0.0.1:58100 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:46:59.848582 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.847838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58116 (local address 127.30.104.253:40203)
0809 00:46:59.847903 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.847906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.848018 (+ 112us) server_negotiation.cc:407] Connection header received
0809 00:46:59.848059 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.848061 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.848076 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.848159 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.848507 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58116: BlockingRecv error: recv got EOF from 127.0.0.1:58116 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 00:46:59.978574 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:46:59.977766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58130 (local address 127.30.104.253:40203)
0809 00:46:59.977846 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 00:46:59.977848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:46:59.977966 (+ 118us) server_negotiation.cc:407] Connection header received
0809 00:46:59.978042 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:46:59.978043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:46:59.978054 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:46:59.978108 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:46:59.978486 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58130: BlockingRecv error: recv got EOF from 127.0.0.1:58130 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 00:47:00.236856 31117 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:00.236044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58146 (local address 127.30.104.253:40203)
0809 00:47:00.236104 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:00.236106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:00.236151 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:47:00.236220 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:00.236221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:00.236231 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:00.236299 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:00.236774 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58146: BlockingRecv error: recv got EOF from 127.0.0.1:58146 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 00:47:00.611104 31282 catalog_manager.cc:1259] Loaded cluster ID: c9c4084cdd4f44afaaa241e3b145a07d
I20260809 00:47:00.611193 31282 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92: loading cluster ID for follower catalog manager: success
I20260809 00:47:00.612114 31282 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92: acquiring CA information for follower catalog manager: success
I20260809 00:47:00.612468 31282 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 777a8fa797b44660a133826e5582dc92: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:47:00.664888 31314 catalog_manager.cc:1259] Loaded cluster ID: c9c4084cdd4f44afaaa241e3b145a07d
I20260809 00:47:00.665004 31314 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd: loading cluster ID for follower catalog manager: success
I20260809 00:47:00.665853 31314 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd: acquiring CA information for follower catalog manager: success
I20260809 00:47:00.666141 31314 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fcc7c52db94f48508a09a1910129f4fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:47:00.752648 31418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:00.750082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58148 (local address 127.30.104.253:40203)
0809 00:47:00.750230 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:00.750232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:00.750256 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:47:00.750339 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:00.750340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:00.750394 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:00.750462 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:00.750606 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.751107 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.751907 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.752101 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.752155 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:00.752296 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:00.752406 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:00.752465 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:00.752513 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 00:47:00.752536 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260809 00:47:00.757580 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:00.754761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51864 (local address 127.30.104.252:32811)
0809 00:47:00.754889 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:00.754891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:00.754955 (+ 64us) server_negotiation.cc:407] Connection header received
0809 00:47:00.755093 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:00.755094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:00.755136 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:00.755201 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:00.755339 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.755887 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.756800 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.756999 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.757067 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:00.757215 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:00.757317 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:00.757372 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:00.757474 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 00:47:00.757493 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260809 00:47:00.762356 31425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:00.759688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.30.104.252:32811)
0809 00:47:00.759778 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:00.759780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:00.759825 (+ 45us) server_negotiation.cc:407] Connection header received
0809 00:47:00.759887 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:00.759888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:00.759896 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:00.759942 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:00.760099 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.760552 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.761544 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.761826 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.761882 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:00.762080 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:00.762114 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:00.762178 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:00.762241 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 00:47:00.762262 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 00:47:00.762660 31418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:00.759316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.30.104.253:40203)
0809 00:47:00.759358 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:00.759360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:00.759573 (+ 213us) server_negotiation.cc:407] Connection header received
0809 00:47:00.759647 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:00.759648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:00.759663 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:00.759712 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:00.759930 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.760666 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.761479 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.761773 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.761877 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:00.762369 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:00.762406 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:00.762462 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:00.762561 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 00:47:00.762586 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 00:47:00.762948 31431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:00.759360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46192 (local address 127.30.104.254:46371)
0809 00:47:00.759494 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:00.759496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:00.759749 (+ 253us) server_negotiation.cc:407] Connection header received
0809 00:47:00.759797 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:00.759798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:00.759858 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:00.759941 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:00.760084 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.760834 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.761883 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:00.762096 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:00.762170 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:00.762452 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:00.762573 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:00.762636 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:00.762815 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 00:47:00.762842 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260809 00:47:00.764664 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 30904
I20260809 00:47:00.775521 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31009
I20260809 00:47:00.787566 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31120
2026-08-09T00:47:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3068 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 00:47:00 dist-test-slave-c12x krb5kdc[31438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 00:47:00 dist-test-slave-c12x krb5kdc[31438](info): set up 2 sockets
Aug 09 00:47:00 dist-test-slave-c12x krb5kdc[31438](info): commencing operation
krb5kdc: starting...
W20260809 00:47:02.018460 31139 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/test-user.keytab.
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T00:47:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T00:47:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.104.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
Entry for principal kudu/127.30.104.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab.
I20260809 00:47:02.150444 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33265
--webserver_interface=127.30.104.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:33265,127.30.104.253:33905,127.30.104.252:34495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 00:47:02.235253 31454 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:47:02.235526 31454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:47:02.235545 31454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:47:02.235569 31454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:47:02.236743 31454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:47:02.236781 31454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:47:02.236796 31454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:47:02.236809 31454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:47:02.236821 31454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:47:02.238368 31454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:33265,127.30.104.253:33905,127.30.104.252:34495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.254.keytab
--principal=kudu/127.30.104.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.254:33265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.104.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:47:02.238605 31454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:47:02.238802 31454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:47:02.244117 31467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:47:02.244359 31454 server_base.cc:1034] running on GCE node
W20260809 00:47:02.244107 31463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:47:02.244107 31464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:47:02.244750 31454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:47:02.245021 31454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:47:02.246182 31454 hybrid_clock.cc:648] HybridClock initialized: now 1786236422246161 us; error 39 us; skew 500 ppm
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:47:02.249054 31454 init.cc:377] Logged in from keytab as kudu/127.30.104.254@KRBTEST.COM (short username kudu)
I20260809 00:47:02.249610 31454 webserver.cc:460] Webserver started at http://127.30.104.254:33507/ using document root <none> and password file <none>
I20260809 00:47:02.249867 31454 fs_manager.cc:359] Metadata directory not provided
I20260809 00:47:02.249917 31454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:47:02.250113 31454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:47:02.251242 31454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/data/instance:
uuid: "753feca18360478b9a827a40838bc5e7"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "5c2dc4f36fa35771d4c3154a840cb216"
server_key_iv: "7baaf0fc8a5132bb79730f46ad79cf0e"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.251668 31454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/wal/instance:
uuid: "753feca18360478b9a827a40838bc5e7"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "5c2dc4f36fa35771d4c3154a840cb216"
server_key_iv: "7baaf0fc8a5132bb79730f46ad79cf0e"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.253132 31454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:47:02.254019 31476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:47:02.254256 31454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 00:47:02.254364 31454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/wal
uuid: "753feca18360478b9a827a40838bc5e7"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "5c2dc4f36fa35771d4c3154a840cb216"
server_key_iv: "7baaf0fc8a5132bb79730f46ad79cf0e"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.254451 31454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:47:02.266202 31454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:47:02.268680 31454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:47:02.268855 31454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:47:02.273723 31454 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.254:33265
I20260809 00:47:02.273764 31553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.254:33265 every 8 connection(s)
I20260809 00:47:02.274175 31454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/data/info.pb
I20260809 00:47:02.275190 31554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:47:02.276767 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31454
I20260809 00:47:02.276877 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-0/wal/instance
I20260809 00:47:02.277096 31139 external_mini_cluster.cc:1428] Setting key 7607eed945897d5bfee93f60ae26983c
I20260809 00:47:02.277138 31554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.104.253@KRBTEST.COM; defaulting to no policy
I20260809 00:47:02.289054 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.277963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33265 (local address 127.0.0.1:43488)
0809 00:47:02.278146 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.278192 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.278940 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.279299 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.279301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.279666 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.279827 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.279832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.281386 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.281387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.282150 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.282153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.282851 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.282852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.282941 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.283660 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.283671 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.285448 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.287377 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.287384 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.287386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.287630 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.287881 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.287882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.287883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.287977 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.288354 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.288359 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.288460 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.288548 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.288796 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":96,"thread_start_us":88,"threads_started":1}
I20260809 00:47:02.289105 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.278142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.30.104.254:33265)
0809 00:47:02.278283 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.278288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.278326 (+ 38us) server_negotiation.cc:407] Connection header received
0809 00:47:02.279144 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.279147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.279170 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.279272 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.279934 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.281279 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.282241 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.282755 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.282840 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.285548 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.285557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.285557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.285565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.287291 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.287724 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.287727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.287727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.287813 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.288049 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.288050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.288050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.288276 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.288365 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.288597 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.288730 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.253@KRBTEST.COM" created.
I20260809 00:47:02.289992 31554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:47:02.290382 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.290264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41508)
0809 00:47:02.290313 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.290359 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 00:47:02.290532 31478 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
I20260809 00:47:02.290973 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.290847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41522)
0809 00:47:02.290902 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.290938 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:47:02.291168 31554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:33905: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab.
Entry for principal kudu/127.30.104.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab.
I20260809 00:47:02.306638 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.253:33905
--webserver_interface=127.30.104.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab
--principal=kudu/127.30.104.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:33265,127.30.104.253:33905,127.30.104.252:34495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:47:02.340420 31554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } attempt: 1
I20260809 00:47:02.340981 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.340843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41532)
0809 00:47:02.340905 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.340941 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 00:47:02.341447 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.341307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41538)
0809 00:47:02.341358 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.341420 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:47:02.341627 31554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:33905: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
W20260809 00:47:02.393368 31567 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:47:02.393644 31567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:47:02.393677 31567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:47:02.393718 31567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:47:02.394888 31567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:47:02.394963 31567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:47:02.394989 31567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:47:02.395020 31567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:47:02.395051 31567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:47:02.396612 31567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:33265,127.30.104.253:33905,127.30.104.252:34495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.253.keytab
--principal=kudu/127.30.104.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.253:33905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.104.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:47:02.396914 31567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:47:02.397148 31567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:47:02.402920 31576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:47:02.402946 31572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:47:02.403118 31567 server_base.cc:1034] running on GCE node
W20260809 00:47:02.402906 31573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:47:02.403497 31567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:47:02.403833 31567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:47:02.404990 31567 hybrid_clock.cc:648] HybridClock initialized: now 1786236422404976 us; error 34 us; skew 500 ppm
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:47:02.407744 31567 init.cc:377] Logged in from keytab as kudu/127.30.104.253@KRBTEST.COM (short username kudu)
I20260809 00:47:02.408267 31567 webserver.cc:460] Webserver started at http://127.30.104.253:42929/ using document root <none> and password file <none>
I20260809 00:47:02.408524 31567 fs_manager.cc:359] Metadata directory not provided
I20260809 00:47:02.408598 31567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:47:02.408746 31567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:47:02.409790 31554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } attempt: 2
I20260809 00:47:02.409869 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/data/instance:
uuid: "f89ad8817ccd4a74908544eb295af29d"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "440b1d20db33f0de259c982e174e2e75"
server_key_iv: "ded30dd2fca417907e44d799fcf6d9ff"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.410223 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.410083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41542)
0809 00:47:02.410148 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.410183 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:47:02.410331 31567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/wal/instance:
uuid: "f89ad8817ccd4a74908544eb295af29d"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "440b1d20db33f0de259c982e174e2e75"
server_key_iv: "ded30dd2fca417907e44d799fcf6d9ff"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.410568 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.410454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41544)
0809 00:47:02.410502 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.410539 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 00:47:02.410704 31554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.253:33905: Network error: Client connection negotiation failed: client connection to 127.30.104.253:33905: connect: Connection refused (error 111)
I20260809 00:47:02.411923 31567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 00:47:02.412708 31585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:47:02.412901 31567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:47:02.412997 31567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/wal
uuid: "f89ad8817ccd4a74908544eb295af29d"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "440b1d20db33f0de259c982e174e2e75"
server_key_iv: "ded30dd2fca417907e44d799fcf6d9ff"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.413076 31567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:47:02.444087 31567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:47:02.446630 31567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:47:02.446792 31567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:47:02.451094 31567 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.253:33905
I20260809 00:47:02.451131 31660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.253:33905 every 8 connection(s)
I20260809 00:47:02.451526 31567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/data/info.pb
I20260809 00:47:02.452443 31661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:47:02.454077 31661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:47:02.454221 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31567
I20260809 00:47:02.454375 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-1/wal/instance
I20260809 00:47:02.454619 31139 external_mini_cluster.cc:1428] Setting key 6e21370af119daf40fb6b2043d64045f
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.104.252@KRBTEST.COM; defaulting to no policy
I20260809 00:47:02.465368 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.454861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.30.104.254:33265)
0809 00:47:02.454917 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.454919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.455142 (+ 223us) server_negotiation.cc:407] Connection header received
0809 00:47:02.455684 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.455685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.455702 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.455777 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.456402 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.457010 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.458485 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.458698 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.458755 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.461550 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.461552 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.461553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.461563 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.463620 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.464284 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.464286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.464286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.464359 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.464638 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.464639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.464639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.464763 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.464820 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.465175 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.465272 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 00:47:02.465615 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.454868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33265 (local address 127.0.0.1:43498)
0809 00:47:02.455030 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.455066 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.455586 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.455830 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.455833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.456085 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.456278 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.456301 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.457107 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.457108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.458400 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.458403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.458781 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.458783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.458945 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.459720 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.459729 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.461435 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.463739 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.463748 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.463751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.464011 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.464446 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.464447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.464447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.464576 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.464847 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.464854 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.465002 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.465137 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.465315 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
I20260809 00:47:02.466315 31661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.104.252@KRBTEST.COM" created.
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:47:02.475106 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.466620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41550)
0809 00:47:02.466680 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.466697 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.466746 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.467049 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.467050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.467242 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.467315 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.467317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.468254 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.468256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.468999 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.469002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.469696 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.469697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.469768 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.470253 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.470258 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.471626 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.473902 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.473903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.473903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.474081 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.474367 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.474368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.474368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.474439 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.474812 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.474813 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.474855 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.474923 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.475030 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 00:47:02.475265 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.466678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.30.104.253:33905)
0809 00:47:02.466872 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.466877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.466901 (+ 24us) server_negotiation.cc:407] Connection header received
0809 00:47:02.466926 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.466928 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.466944 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.467032 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.467406 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.468167 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.469084 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.469610 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.469689 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.471716 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.471726 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.471727 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.471735 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.473806 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.474169 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.474173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.474173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.474248 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.474515 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.474515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.474516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.474732 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.474828 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.474973 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.475109 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":160,"thread_start_us":112,"threads_started":1}
I20260809 00:47:02.476276 31661 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:47:02.476713 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.476580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48272)
0809 00:47:02.476632 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.476691 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 00:47:02.476897 31591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
I20260809 00:47:02.477329 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.477245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48288)
0809 00:47:02.477287 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.477313 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 00:47:02.477511 31661 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:34495: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.104.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab.
Entry for principal kudu/127.30.104.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab.
I20260809 00:47:02.483978 31139 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxSoRG4/build/release/bin/kudu
/tmp/dist-test-taskxSoRG4/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.252:34495
--webserver_interface=127.30.104.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab
--principal=kudu/127.30.104.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.104.212:33571
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.104.254:33265,127.30.104.253:33905,127.30.104.252:34495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 00:47:02.501906 31554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } attempt: 3
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:47:02.509634 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.502374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41564 (local address 127.30.104.253:33905)
0809 00:47:02.502462 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.502465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.502478 (+ 13us) server_negotiation.cc:407] Connection header received
0809 00:47:02.502539 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.502540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.502554 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.502629 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.503014 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.503623 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.504400 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.504648 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.504735 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.506956 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.506958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.506958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.506967 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.508478 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.508820 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.508821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.508822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.508878 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.509101 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.509102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.509102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.509213 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.509294 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.509452 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.509530 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 00:47:02.509634 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.502284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41564)
0809 00:47:02.502351 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.502366 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.502470 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.502635 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.502636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.502836 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.502920 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.502922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.503710 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.503712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.504296 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.504298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.504734 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.504735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.504826 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.505336 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.505341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.506800 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.508568 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.508569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.508570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.508748 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.508956 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.508957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.508957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.509022 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.509285 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.509286 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.509336 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.509421 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.509526 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:47:02.510082 31554 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:47:02.510452 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.510338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48298)
0809 00:47:02.510384 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.510413 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 00:47:02.510812 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.510731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48304)
0809 00:47:02.510769 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.510792 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 00:47:02.510972 31554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:34495: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
I20260809 00:47:02.526757 31661 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } attempt: 1
I20260809 00:47:02.527268 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.527131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48308)
0809 00:47:02.527196 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.527237 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 00:47:02.527767 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.527624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48312)
0809 00:47:02.527707 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.527742 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260809 00:47:02.527909 31661 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:34495: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
I20260809 00:47:02.542150 31554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } attempt: 1
I20260809 00:47:02.542708 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.542565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48314)
0809 00:47:02.542632 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.542673 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 00:47:02.543072 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.542970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48322)
0809 00:47:02.543018 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.543050 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 00:47:02.543167 31554 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:34495: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
W20260809 00:47:02.570901 31670 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 00:47:02.571197 31670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 00:47:02.571228 31670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 00:47:02.571270 31670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 00:47:02.572517 31670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 00:47:02.572594 31670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 00:47:02.572618 31670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 00:47:02.572639 31670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 00:47:02.572666 31670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 00:47:02.574247 31670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.104.212:33571
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.104.254:33265,127.30.104.253:33905,127.30.104.252:34495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/krb5kdc/kudu_127.30.104.252.keytab
--principal=kudu/127.30.104.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.104.252:34495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.104.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 08 Aug 2026 23:50:12 UTC on ccycloud.quasar-fckmho.root.comops.site
build id 4575724
I20260809 00:47:02.574544 31670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 00:47:02.574785 31670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 00:47:02.579907 31679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:47:02.579903 31681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 00:47:02.580013 31678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 00:47:02.580262 31670 server_base.cc:1034] running on GCE node
I20260809 00:47:02.580595 31670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 00:47:02.580917 31670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 00:47:02.582088 31670 hybrid_clock.cc:648] HybridClock initialized: now 1786236422582069 us; error 28 us; skew 500 ppm
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 00:47:02.584935 31670 init.cc:377] Logged in from keytab as kudu/127.30.104.252@KRBTEST.COM (short username kudu)
I20260809 00:47:02.585491 31670 webserver.cc:460] Webserver started at http://127.30.104.252:37219/ using document root <none> and password file <none>
I20260809 00:47:02.585741 31670 fs_manager.cc:359] Metadata directory not provided
I20260809 00:47:02.585815 31670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 00:47:02.585991 31670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 00:47:02.587164 31670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/data/instance:
uuid: "7e20dc6101dc420382487f0cf19f6833"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "e8d76b88e623431d5a364211ef6eac8b"
server_key_iv: "14869636600c684ba23f49fed92b708f"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.587630 31670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/wal/instance:
uuid: "7e20dc6101dc420382487f0cf19f6833"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "e8d76b88e623431d5a364211ef6eac8b"
server_key_iv: "14869636600c684ba23f49fed92b708f"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.589184 31670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:47:02.590047 31692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 00:47:02.590332 31670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 00:47:02.590476 31670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/data,/tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/wal
uuid: "7e20dc6101dc420382487f0cf19f6833"
format_stamp: "Formatted at 2026-08-09 00:47:02 on dist-test-slave-c12x"
server_key: "e8d76b88e623431d5a364211ef6eac8b"
server_key_iv: "14869636600c684ba23f49fed92b708f"
server_key_version: "encryptionkey@0"
I20260809 00:47:02.590557 31670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 00:47:02.596076 31661 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } attempt: 2
I20260809 00:47:02.596496 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.596356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48334)
0809 00:47:02.596419 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.596460 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:47:02.596889 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.596793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48338)
0809 00:47:02.596837 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.596866 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 00:47:02.597005 31661 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.104.252:34495: Network error: Client connection negotiation failed: client connection to 127.30.104.252:34495: connect: Connection refused (error 111)
I20260809 00:47:02.600960 31670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 00:47:02.603232 31670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 00:47:02.603389 31670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 00:47:02.607203 31670 rpc_server.cc:307] RPC server started. Bound to: 127.30.104.252:34495
I20260809 00:47:02.607278 31764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.104.252:34495 every 8 connection(s)
I20260809 00:47:02.607661 31670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/data/info.pb
I20260809 00:47:02.608704 31765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 00:47:02.610059 31139 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxSoRG4/build/release/bin/kudu as pid 31670
I20260809 00:47:02.610179 31139 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxSoRG4/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786236131192310-31139-0/minicluster-data/master-2/wal/instance
I20260809 00:47:02.610399 31139 external_mini_cluster.cc:1428] Setting key c2fd41a2cc096937701c683bc54486a1
I20260809 00:47:02.610564 31765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:47:02.618309 31554 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } attempt: 2
I20260809 00:47:02.621701 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.611211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.254:33265 (local address 127.0.0.1:43506)
0809 00:47:02.611331 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.611424 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.611835 (+ 411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.612094 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.612096 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.612368 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.612534 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.612539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.613247 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.613248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.614122 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.614124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.614564 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.614565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.614834 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.615630 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.615643 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.617620 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.619896 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.619905 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.619908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.620195 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.620426 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.620427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.620427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.620569 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.621004 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.621011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.621183 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.621291 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.621476 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":69,"threads_started":1}
I20260809 00:47:02.622296 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.611317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43506 (local address 127.30.104.254:33265)
0809 00:47:02.611370 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.611372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.611509 (+ 137us) server_negotiation.cc:407] Connection header received
0809 00:47:02.611976 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.611977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.611990 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.612060 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.612632 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.613168 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.614215 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.614452 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.614526 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.617732 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.617734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.617734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.617744 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.619657 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.620277 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.620278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.620278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.620346 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.620660 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.620661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.620662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.620887 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.620970 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.621887 (+ 917us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.621978 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 00:47:02.622614 31768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.610975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48344 (local address 127.30.104.252:34495)
0809 00:47:02.611145 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.611154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.611176 (+ 22us) server_negotiation.cc:407] Connection header received
0809 00:47:02.611834 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.611844 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.611869 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.611962 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.612639 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.613621 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.614425 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.614873 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.614935 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.618265 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.618273 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.618274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.618281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.620428 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.621035 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.621038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.621039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.621108 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.621331 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.621332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.621332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.621516 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.622345 (+ 829us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.622405 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.622496 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260809 00:47:02.622884 31765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.254@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:47:02.628186 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.618617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48346)
0809 00:47:02.618661 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.618673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.618713 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.619223 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.619224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.619357 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.619428 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.619430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.620475 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.620476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.621380 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.621383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.621853 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.621854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.621921 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.622286 (+ 365us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.622292 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.624306 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.626937 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.626938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.626939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.627185 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.627403 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.627404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.627405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.627531 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.627831 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.627832 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.627878 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.627964 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.628071 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 00:47:02.628216 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.618765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48346 (local address 127.30.104.252:34495)
0809 00:47:02.619058 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.619060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.619070 (+ 10us) server_negotiation.cc:407] Connection header received
0809 00:47:02.619091 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.619092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.619108 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.619546 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.619586 (+ 40us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.620343 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.621457 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.621763 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.622098 (+ 335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.624403 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.624405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.624406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.624414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.626853 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.627275 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.627277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.627277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.627334 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.627614 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.627615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.627615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.627733 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.627834 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.628004 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.628084 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":186,"threads_started":1}
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.254@KRBTEST.COM
I20260809 00:47:02.630707 31554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7: Bootstrap starting.
I20260809 00:47:02.631637 31554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7: Neither blocks nor log segments found. Creating new log.
I20260809 00:47:02.631979 31554 log.cc:824] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7: Log is configured to *not* fsync() on all Append() calls
I20260809 00:47:02.632061 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.623980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41570 (local address 127.30.104.253:33905)
0809 00:47:02.624091 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.624094 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.624102 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:47:02.624128 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.624129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.624141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.624217 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.624543 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.625186 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.626097 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.626414 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.626474 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.628858 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.628860 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.628861 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.628868 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.630627 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.631053 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.631054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.631054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.631131 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.631360 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.631361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.631362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.631569 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.631643 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.631863 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.631959 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 00:47:02.632381 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.623171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.253:33905 (local address 127.0.0.1:41570)
0809 00:47:02.623224 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.623236 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.623273 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.624235 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.624236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.624404 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.624463 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.624465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.625286 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.625287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.625998 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.626000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.626483 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.626484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.626556 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.626963 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.626969 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.628754 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.630739 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.630740 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.630741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.630966 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.631217 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.631218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.631219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.631288 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.631645 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.631646 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.631741 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.631811 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.631916 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 00:47:02.632822 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.623888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.30.104.254:33265)
0809 00:47:02.623945 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.623948 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.623957 (+ 9us) server_negotiation.cc:407] Connection header received
0809 00:47:02.623979 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.623980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.623991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.624055 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.624749 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.625488 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.626441 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.627030 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.627110 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.629803 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.629805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.629805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.629812 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.631494 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.631862 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.631863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.631863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.631921 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.632120 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.632121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.632121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.632268 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.632405 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.632646 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.632732 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 00:47:02.632783 31765 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } has no permanent_uuid. Determining permanent_uuid...
I20260809 00:47:02.633030 31554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7: No bootstrap required, opened a new log
I20260809 00:47:02.635133 31554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } }
I20260809 00:47:02.635303 31554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753feca18360478b9a827a40838bc5e7, State: Initialized, Role: FOLLOWER
I20260809 00:47:02.635411 31554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } }
I20260809 00:47:02.635795 31777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.635874 31554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:47:02.635962 31777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:47:02.638003 31793 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:47:02.638067 31793 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.252@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.104.253@KRBTEST.COM
I20260809 00:47:02.638999 31454 master_runner.cc:186] Error getting master registration for 127.30.104.253:33905: OK, instance_id { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" instance_seqno: 1786236422449819 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:47:02.642004 31454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:47:02.643965 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.633596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41582 (local address 127.30.104.253:33905)
0809 00:47:02.633651 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.633653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.633701 (+ 48us) server_negotiation.cc:407] Connection header received
0809 00:47:02.633790 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.633791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.633801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.633866 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.634485 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.635182 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.636179 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.636532 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.636959 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.640473 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.640474 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.640475 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.640482 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.642287 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.643015 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.643017 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.643017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.643083 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.643322 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.643323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.643323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.643514 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.643701 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.643757 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.643845 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 00:47:02.644608 31769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.633575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48350)
0809 00:47:02.633699 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.633708 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.633750 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.633886 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.633887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.634070 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.634117 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.634120 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.635142 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.635143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.635947 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.635950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.636394 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.636395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.636442 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.636922 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.636927 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.638505 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.643372 (+ 4867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.643372 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.643373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.643600 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.643802 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.643803 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.643803 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.643856 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.644253 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.644253 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.644296 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.644377 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.644498 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 00:47:02.644795 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.633663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.30.104.252:34495)
0809 00:47:02.633705 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.633707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.633738 (+ 31us) server_negotiation.cc:407] Connection header received
0809 00:47:02.633817 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.633818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.633825 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.634191 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.634217 (+ 26us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.635015 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.636046 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.636320 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.636398 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.638607 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.638608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.638609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.638618 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.643272 (+ 4654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.643679 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.643680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.643680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.643740 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.643913 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.643914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.643915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.644104 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.644199 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.644400 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.644738 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":247,"server-negotiator.queue_time_us":21}
I20260809 00:47:02.645960 31139 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 00:47:02.646543 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833: Bootstrap starting.
I20260809 00:47:02.646975 31517 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "753feca18360478b9a827a40838bc5e7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43514
I20260809 00:47:02.647135 31517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 00:47:02.647192 31517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:47:02.647310 31765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833: Neither blocks nor log segments found. Creating new log.
I20260809 00:47:02.647651 31765 log.cc:824] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833: Log is configured to *not* fsync() on all Append() calls
I20260809 00:47:02.648427 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833: No bootstrap required, opened a new log
I20260809 00:47:02.648496 31517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } }
I20260809 00:47:02.648770 31517 leader_election.cc:288] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [CANDIDATE]: Term 1 election: Requested vote from peers f89ad8817ccd4a74908544eb295af29d (127.30.104.253:33905), 7e20dc6101dc420382487f0cf19f6833 (127.30.104.252:34495)
I20260809 00:47:02.650736 31765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } }
I20260809 00:47:02.650899 31765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e20dc6101dc420382487f0cf19f6833, State: Initialized, Role: FOLLOWER
I20260809 00:47:02.650988 31765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } }
I20260809 00:47:02.651299 31810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.651350 31727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "753feca18360478b9a827a40838bc5e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7e20dc6101dc420382487f0cf19f6833"
I20260809 00:47:02.651317 31765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:47:02.651619 31727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:47:02.651577 31810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:47:02.653210 31727 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753feca18360478b9a827a40838bc5e7 in term 1.
I20260809 00:47:02.653463 31484 leader_election.cc:302] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 753feca18360478b9a827a40838bc5e7, 7e20dc6101dc420382487f0cf19f6833; no voters:
I20260809 00:47:02.655473 31777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 00:47:02.655764 31777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [term 1 LEADER]: Becoming Leader. State: Replica: 753feca18360478b9a827a40838bc5e7, State: Running, Role: LEADER
I20260809 00:47:02.655939 31777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } }
I20260809 00:47:02.656951 31777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 753feca18360478b9a827a40838bc5e7. Latest consensus state: current_term: 1 leader_uuid: "753feca18360478b9a827a40838bc5e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.657104 31777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: This master's current role is: LEADER
W20260809 00:47:02.658324 31825 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:47:02.658429 31825 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:47:02.659049 31832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 00:47:02.659373 31832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 00:47:02.658667 31670 master_runner.cc:186] Error getting master registration for 127.30.104.253:33905: OK, instance_id { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" instance_seqno: 1786236422449819 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 00:47:02.661227 31670 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 00:47:02.662377 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.650612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.30.104.252:34495)
0809 00:47:02.650646 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.650648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.653738 (+ 3090us) server_negotiation.cc:407] Connection header received
0809 00:47:02.653837 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.653838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.653848 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.653877 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.654600 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.655301 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.656219 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.656849 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.656894 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.658822 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.658823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.658824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.658831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.660861 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.661335 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.661337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.661338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.661400 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.661688 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.661689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.661690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.661835 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.661938 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.662180 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.662275 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 00:47:02.667313 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.650233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43528 (local address 127.30.104.254:33265)
0809 00:47:02.650287 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.650289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.650775 (+ 486us) server_negotiation.cc:407] Connection header received
0809 00:47:02.650849 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.650850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.650863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.650893 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.652654 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.653406 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.655687 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.655979 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.656027 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.663042 (+ 7015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.663044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.663044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.663052 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.664935 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.666089 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.666090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.666091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.666150 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.666349 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.666350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.666351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.666486 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.666600 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.667136 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.667219 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 00:47:02.668386 31727 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [term 1 FOLLOWER]: Refusing update from remote peer 753feca18360478b9a827a40838bc5e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:47:02.668905 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.650415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.30.104.253:33905)
0809 00:47:02.650475 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.650477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.651781 (+ 1304us) server_negotiation.cc:407] Connection header received
0809 00:47:02.651876 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.651877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.651887 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.651916 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.652659 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.653438 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.656507 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.657332 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.657393 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.663362 (+ 5969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.663364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.663364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.663372 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.665689 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.667892 (+ 2203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.667893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.667893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.667958 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.668189 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.668190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.668190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.668331 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.668412 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.668724 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.668814 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 00:47:02.668830 31826 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:47:02.670933 31810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 753feca18360478b9a827a40838bc5e7. Latest consensus state: current_term: 1 leader_uuid: "753feca18360478b9a827a40838bc5e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.671128 31810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:47:02.671336 31777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "753feca18360478b9a827a40838bc5e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.671427 31777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: This master's current role is: LEADER
I20260809 00:47:02.671610 31810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "753feca18360478b9a827a40838bc5e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.671700 31810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:47:02.672101 31834 mvcc.cc:205] Tried to move back new op lower bound from 7316424387247095808 to 7316424387200974848. Current Snapshot: MvccSnapshot[applied={T|T < 7316424387247095808}]
I20260809 00:47:02.672379 31832 catalog_manager.cc:1347] Generated new cluster ID: f7ac6718d907411d85d1ffdb81094936
I20260809 00:47:02.672421 31832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 00:47:02.675920 31835 mvcc.cc:205] Tried to move back new op lower bound from 7316424387247095808 to 7316424387200974848. Current Snapshot: MvccSnapshot[applied={T|T < 7316424387247095808}]
I20260809 00:47:02.688140 31661 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } attempt: 3
Aug 09 00:47:02 dist-test-slave-c12x krb5kdc[31438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786236422, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.104.253@KRBTEST.COM for kudu/127.30.104.252@KRBTEST.COM
I20260809 00:47:02.693953 31832 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 00:47:02.696344 31832 catalog_manager.cc:1504] Loading token signing keys...
I20260809 00:47:02.697549 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.688493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.104.252:34495 (local address 127.0.0.1:48370)
0809 00:47:02.688633 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 00:47:02.688645 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 00:47:02.688692 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 00:47:02.689003 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 00:47:02.689004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 00:47:02.689242 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 00:47:02.689295 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.689297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.690326 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.690327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.691136 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 00:47:02.691139 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.691670 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.691671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 00:47:02.691729 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.692252 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 00:47:02.692258 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 00:47:02.694106 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 00:47:02.696246 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.696247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.696248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.696491 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.696812 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 00:47:02.696813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 00:47:02.696814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 00:47:02.696885 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 00:47:02.697209 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 00:47:02.697210 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 00:47:02.697267 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 00:47:02.697328 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 00:47:02.697441 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260809 00:47:02.698079 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:02.688794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48370 (local address 127.30.104.252:34495)
0809 00:47:02.688837 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.688838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.688846 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:47:02.688889 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.688890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.688900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.688951 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.689395 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.690217 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.691214 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:02.691587 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:02.691661 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:02.694340 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 00:47:02.694341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 00:47:02.694342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 00:47:02.694361 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 00:47:02.696132 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.696652 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.696653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.696653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.696727 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 00:47:02.696964 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 00:47:02.696964 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 00:47:02.696965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 00:47:02.697112 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 00:47:02.697171 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:02.697903 (+ 732us) server_negotiation.cc:299] Negotiation successful
0809 00:47:02.697990 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 00:47:02.700091 31661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d: Bootstrap starting.
I20260809 00:47:02.700894 31661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d: Neither blocks nor log segments found. Creating new log.
I20260809 00:47:02.701198 31661 log.cc:824] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d: Log is configured to *not* fsync() on all Append() calls
I20260809 00:47:02.703965 31832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7: Generated new TSK 0
I20260809 00:47:02.704442 31832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 00:47:02.708273 31661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d: No bootstrap required, opened a new log
I20260809 00:47:02.710616 31661 raft_consensus.cc:348] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } }
I20260809 00:47:02.710785 31661 raft_consensus.cc:729] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89ad8817ccd4a74908544eb295af29d, State: Initialized, Role: FOLLOWER
I20260809 00:47:02.710870 31661 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } }
I20260809 00:47:02.711259 31661 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 00:47:02.712982 31624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "753feca18360478b9a827a40838bc5e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f89ad8817ccd4a74908544eb295af29d"
I20260809 00:47:02.713126 31624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [term 0 FOLLOWER]: Advancing to term 1
I20260809 00:47:02.714138 31624 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 753feca18360478b9a827a40838bc5e7 in term 1.
I20260809 00:47:02.717682 31623 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [term 1 FOLLOWER]: Refusing update from remote peer 753feca18360478b9a827a40838bc5e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 00:47:02.718127 31826 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 00:47:02.718163 31840 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.718371 31840 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:47:02.721040 31840 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 753feca18360478b9a827a40838bc5e7. Latest consensus state: current_term: 1 leader_uuid: "753feca18360478b9a827a40838bc5e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.721139 31840 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [sys.catalog]: This master's current role is: FOLLOWER
I20260809 00:47:02.721282 31860 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "753feca18360478b9a827a40838bc5e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:02.721402 31860 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d [sys.catalog]: This master's current role is: FOLLOWER
W20260809 00:47:02.722036 31856 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 00:47:02.722105 31856 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 00:47:02.728204 31858 mvcc.cc:205] Tried to move back new op lower bound from 7316424387390406656 to 7316424387200974848. Current Snapshot: MvccSnapshot[applied={T|T < 7316424387247095808}]
W20260809 00:47:02.739588 31771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.738310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48372 (local address 127.30.104.252:34495)
0809 00:47:02.738365 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.738368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.738762 (+ 394us) server_negotiation.cc:407] Connection header received
0809 00:47:02.738888 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.738889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.738902 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.738936 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.739504 (+ 568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48372: BlockingRecv error: recv got EOF from 127.0.0.1:48372 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 00:47:02.741029 31558 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.740007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.30.104.254:33265)
0809 00:47:02.740067 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.740069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.740429 (+ 360us) server_negotiation.cc:407] Connection header received
0809 00:47:02.740537 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.740538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.740551 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.740585 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.740954 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43530: BlockingRecv error: recv got EOF from 127.0.0.1:43530 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 00:47:02.741909 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.740303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41588 (local address 127.30.104.253:33905)
0809 00:47:02.740345 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.740348 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.741346 (+ 998us) server_negotiation.cc:407] Connection header received
0809 00:47:02.741453 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.741455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.741465 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.741497 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.741843 (+ 346us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41588: BlockingRecv error: recv got EOF from 127.0.0.1:41588 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:47:02.744447 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.743634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41596 (local address 127.30.104.253:33905)
0809 00:47:02.743673 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.743675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.743870 (+ 195us) server_negotiation.cc:407] Connection header received
0809 00:47:02.743987 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.743988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.744009 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.744052 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.744395 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41596: BlockingRecv error: recv got EOF from 127.0.0.1:41596 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 00:47:02.747712 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.746860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.30.104.253:33905)
0809 00:47:02.746896 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.746898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.747143 (+ 245us) server_negotiation.cc:407] Connection header received
0809 00:47:02.747249 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.747250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.747261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.747292 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.747661 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41602: BlockingRecv error: recv got EOF from 127.0.0.1:41602 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:47:02.751955 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.750985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.30.104.253:33905)
0809 00:47:02.751056 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.751058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.751333 (+ 275us) server_negotiation.cc:407] Connection header received
0809 00:47:02.751483 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.751484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.751496 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.751529 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.751903 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41618: BlockingRecv error: recv got EOF from 127.0.0.1:41618 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 00:47:02.758533 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.757488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.30.104.253:33905)
0809 00:47:02.757550 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.757553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.757810 (+ 257us) server_negotiation.cc:407] Connection header received
0809 00:47:02.757924 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.757925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.757937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.757971 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.758468 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41632: BlockingRecv error: recv got EOF from 127.0.0.1:41632 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 00:47:02.769161 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.768169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.30.104.253:33905)
0809 00:47:02.768217 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.768220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.768487 (+ 267us) server_negotiation.cc:407] Connection header received
0809 00:47:02.768605 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.768607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.768619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.768653 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.769080 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41644: BlockingRecv error: recv got EOF from 127.0.0.1:41644 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 00:47:02.787755 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.786773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41658 (local address 127.30.104.253:33905)
0809 00:47:02.786819 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.786821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.787086 (+ 265us) server_negotiation.cc:407] Connection header received
0809 00:47:02.787198 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.787200 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.787212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.787243 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.787691 (+ 448us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41658: BlockingRecv error: recv got EOF from 127.0.0.1:41658 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 00:47:02.822237 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.821238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.30.104.253:33905)
0809 00:47:02.821287 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.821290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.821566 (+ 276us) server_negotiation.cc:407] Connection header received
0809 00:47:02.821676 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.821677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.821690 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.821722 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.822172 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41660: BlockingRecv error: recv got EOF from 127.0.0.1:41660 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 00:47:02.888738 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:02.887914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41668 (local address 127.30.104.253:33905)
0809 00:47:02.887982 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:02.887984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:02.888116 (+ 132us) server_negotiation.cc:407] Connection header received
0809 00:47:02.888194 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:02.888195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:02.888207 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:02.888271 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:02.888676 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41668: BlockingRecv error: recv got EOF from 127.0.0.1:41668 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260809 00:47:03.018829 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:03.018102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41682 (local address 127.30.104.253:33905)
0809 00:47:03.018177 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:03.018179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:03.018285 (+ 106us) server_negotiation.cc:407] Connection header received
0809 00:47:03.018333 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:03.018334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:03.018345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:03.018419 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:03.018748 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41682: BlockingRecv error: recv got EOF from 127.0.0.1:41682 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 00:47:03.100960 31826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "753feca18360478b9a827a40838bc5e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753feca18360478b9a827a40838bc5e7" member_type: VOTER last_known_addr { host: "127.30.104.254" port: 33265 } } peers { permanent_uuid: "f89ad8817ccd4a74908544eb295af29d" member_type: VOTER last_known_addr { host: "127.30.104.253" port: 33905 } } peers { permanent_uuid: "7e20dc6101dc420382487f0cf19f6833" member_type: VOTER last_known_addr { host: "127.30.104.252" port: 34495 } } }
I20260809 00:47:03.101151 31826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 753feca18360478b9a827a40838bc5e7 [sys.catalog]: This master's current role is: LEADER
W20260809 00:47:03.276854 31666 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 00:47:03.276196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.30.104.253:33905)
0809 00:47:03.276274 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:03.276277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:03.276323 (+ 46us) server_negotiation.cc:407] Connection header received
0809 00:47:03.276373 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:03.276374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:03.276384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:03.276451 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 00:47:03.276782 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41692: BlockingRecv error: recv got EOF from 127.0.0.1:41692 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 00:47:03.658926 31825 catalog_manager.cc:1259] Loaded cluster ID: f7ac6718d907411d85d1ffdb81094936
I20260809 00:47:03.659010 31825 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833: loading cluster ID for follower catalog manager: success
I20260809 00:47:03.659890 31825 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833: acquiring CA information for follower catalog manager: success
I20260809 00:47:03.660228 31825 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e20dc6101dc420382487f0cf19f6833: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:47:03.722672 31856 catalog_manager.cc:1259] Loaded cluster ID: f7ac6718d907411d85d1ffdb81094936
I20260809 00:47:03.722774 31856 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d: loading cluster ID for follower catalog manager: success
I20260809 00:47:03.723701 31856 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d: acquiring CA information for follower catalog manager: success
I20260809 00:47:03.724009 31856 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f89ad8817ccd4a74908544eb295af29d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 00:47:03.793563 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:03.790462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41702 (local address 127.30.104.253:33905)
0809 00:47:03.790622 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:03.790624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:03.790686 (+ 62us) server_negotiation.cc:407] Connection header received
0809 00:47:03.790740 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:03.790741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:03.790804 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:03.790888 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:03.791115 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.791744 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.792576 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.792815 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.792880 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:03.793014 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:03.793150 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:03.793221 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:03.793337 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 00:47:03.793362 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260809 00:47:03.798157 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:03.795546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.30.104.252:34495)
0809 00:47:03.795682 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:03.795684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:03.795698 (+ 14us) server_negotiation.cc:407] Connection header received
0809 00:47:03.795784 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:03.795785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:03.795825 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:03.795897 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:03.796005 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.796579 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.797330 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.797575 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.797635 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:03.797771 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:03.797894 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:03.797958 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:03.797987 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 00:47:03.798007 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260809 00:47:03.803009 31976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:03.800028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43542 (local address 127.30.104.254:33265)
0809 00:47:03.800238 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:03.800241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:03.800249 (+ 8us) server_negotiation.cc:407] Connection header received
0809 00:47:03.800394 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:03.800395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:03.800455 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:03.800565 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:03.800716 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.801369 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.802119 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.802341 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.802414 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:03.802623 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:03.802730 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:03.802802 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:03.802860 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 00:47:03.802889 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":67,"threads_started":1}
I20260809 00:47:03.803468 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:03.800324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41708 (local address 127.30.104.253:33905)
0809 00:47:03.800383 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:03.800385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:03.800760 (+ 375us) server_negotiation.cc:407] Connection header received
0809 00:47:03.800857 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:03.800859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:03.800873 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:03.800922 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:03.801083 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.801563 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.802614 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.802910 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.802984 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:03.803166 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:03.803198 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:03.803223 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:03.803319 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 00:47:03.803343 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 00:47:03.804499 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 00:47:03.800167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.30.104.252:34495)
0809 00:47:03.800279 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 00:47:03.800282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 00:47:03.800715 (+ 433us) server_negotiation.cc:407] Connection header received
0809 00:47:03.800779 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 00:47:03.800780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 00:47:03.800790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 00:47:03.800845 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 00:47:03.801678 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.802415 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.803473 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 00:47:03.803955 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 00:47:03.804021 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 00:47:03.804233 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 00:47:03.804269 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 00:47:03.804331 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 00:47:03.804394 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 00:47:03.804416 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 00:47:03.804915 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31454
I20260809 00:47:03.818675 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31567
I20260809 00:47:03.831660 31139 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxSoRG4/build/release/bin/kudu with pid 31670
2026-08-09T00:47:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3046 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6115 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (292654 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST